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INTRODUCTION 


BACKGROUND 
The  Threat  Situation 

IR  (infrared)  guided  missiles  have  been  developed  to  such  a  degree  that  they  have 
become  a  serious  threat  to  aircraft  in  flight.  Detection  and  tracking  systems  that  rely  on  the 
IR  emissions  of  aircraft  are  an  important  threat  consideration.  Newer,  more  sophisticated 
IR  guidance,  detection,  and  tracking  techniques  that  will  very  likely  increase  the  threat 
significantly  in  the  future  are  being  developed. 

Every  effort  is  being  made  to  protect  friendly  aircraft  against  this  threat  by  controlling 
and  reducing  aircraft  IR  emissions  and  by  improving  active  countermeasures  techniques. 
Alternatively,  the  U.S.  military  community  is  pursuing  the  improvement  of  IR  weaponry 
for  defense  against  enemy  aircraft.  Therefore,  the  requirement  exists  for  information  on 
the  IR  emissions  of  aircraft. 

Needs  of  the  Infrared  Community 

Several  facilities  for  the  measurement  of  aircraft  IR  signatures  are  maintained  by 
various  government  agencies  and  by  some  private  companies.  However,  such  measurements 
are  costly  and  time-consuming.  Therefore,  the  reported  results  of  an  aircraft  signature 
measurement  program  are  usually  values  taken  during  a  one-shot  effort  in  the  sense  that  the 
aircraft  and  measurement  equipment  are  positioned  one  time  only  for  each  desired  measure¬ 
ment  configuration.  This  leaves  unanswered  the  question  of  how  much  difference  might 
occur  if  data  were  collected  from  an  attempt  to  repeat  the  setup  on  a  different  occasion. 
Likewise,  discrepancies  occur  when  several  agencies  measure  one  particular  orientation  of 
the  same  aircraft.  This  can  happen  even  if  the  various  measuring  equipments  are  colocated. 

A  reported  value  is  customarily  an  average  of  several  readings  taken  sequentially  with 
one  instrument  (radiometer,  spectrometer,  interferometer).  Even  the  variability  of  these 
sequential  data  samples  has  not  been  the  subject  of  a  specific  study.  In  short,  the  costliness 
of  the  measurement  process  has  resulted  in  the  fact  that  no  full-scale  uncertainty  analysis 
has  been  performed  on  the  aircraft  IR  signature  measurement  process.  Data  collected  by  the 
measurement  facilities  are  reported  to  the  IR  community  without  associated  qualifying 
measurement  uncertainty  brackets. 

Because  of  the  difficulties  involved  in  the  direct  measurement  of  aircraft  IR  signatures, 
an  alternative  approach  to  accumulating  descriptive  information  is  being  undertaken.  This 
is  the  calculation  of  IR  emission  data  by  computer  programs  which  model  the  thermal 
processes  characteristic  of  operating  aircraft.  This  approach  is  in  a  developing  state,  and 
there  is  a  need  to  compare  computer  outputs  with  reliable,  well-qualified  measured  data. 

An  analysis  of  aircraft  IR  signature  measurement  uncertainty  would  benefit  threat 
designers,  developers  of  suppression  and  countermeasures  techniques,  and  writers  of  predic¬ 
tive  theoretical  computer  models,  as  well  as  the  operators  of  the  measurement  facilities. 
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The  Role  of  JIRS 

The  JIRS  (Joint  Intrareci  Standards)  Working  Group  has  been  organized  to  “effect  the 
improvement  of  the  quality  of  infrared  (IR)  measurements  and  their  results  that  pertain 
to  the  IR  susceptibility  of  aircraft”.*  The  JIRS  Working  Group  is  set  up  under  the  Counter¬ 
measures  Subgroup  of  JTCG/AS  (Joint  Technical  Coordinating  Group  on  Aircratt  Sur¬ 
vivability). 

In  fulfillment  of  its  charter  to  improve  the  quality  and  usability  of  IR  data  pertinent 
to  aircraft  susceptibility,  JIRS  has  sponsored  the  measurement  variability  analysis 
described  in  this  report.  This  effort  is  designed  to  be  integrated  into  the  overall  JIRS 
program  of  quantifying  measurement  uncertainty  and  comparing  well-qualified  measure¬ 
ment  data  with  computer  model  aircraft  signature  predictions. 

Scope  of  This  Effort 

Aircraft  IR  signature  measurement  uncertainty  is  the  result  of  several  factors.  A 
spectrum  of  possibilities  arises  from  the  interplay  of  all  the  parameters  inherent  in  the 
measurement  procedures  and  equipment.  In  addition,  the  effects  of  atmosphere,  back¬ 
ground,  and  ambiguity  in  engine  operating  conditions  must  be  considered.  Finally,  the 
measurement  difficulties  are  severely  impacted  by  the  level  of  error  tolerance  to  which  the 
constantly  changing  variables  associated  with  in-flight  aircraft  are  measured. 

To  properly  address  the  problem  of  quantifying  signature  measurement  uncertainty, 
all  the  significant  contributors  should  be  isolated  and  analyzed  separately,  so  that  the  effect 
of  each  can  be  uniquely  determined.  The  effort  described  in  this  report  represents  one 
step  of  the  needed  total  uncertainty  study. 

The  initial  phase  of  a  complete  signature  measurement  uncertainty  study  must  consist 
of  quantification  of  the  measurement  error  factors  generic  to  the  measurement  equipments 
and  procedures  themselves.  Several  types  of  measurement  devices  might  be  chosen  for  study 
including  various  spectrometers,  radiometers,  interferometers,  and  imaging  devices.  For 
this  beginning  effort,  the  Michelson  interferometer  was  selected. 

Three  equipments  based  on  similar  interferometers  are  currently  in  use  at  three  major 
measurement  facilities.  These  are: 

NWC,  China  Lake.  CA  General  Dynamics  Model  No.  PFS-101 

(w/o  optics,  software) 

OMFW,  White  Sands,  NM  General  Dynamics  Model  No.  PFS-201 

(w/o  software) 

General  Dynamics.  Pomona,  CA  General  Dynamics  Model  No.  PFS-201 

The  Michelson  interferometers  used  by  the  Optica!  Signatures  Branch.  Flectro-Optics 
Division  of  NWC  (Naval  Weapons  Center).  China  Lake.  California,  and  the  Fleet ronic 

•Charier  for  the  Joint  I  lit  ruled  Standards  Working  (iioup  tl)RAI  T).  September.  I97B. 
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Warfare  Laboratory,  OMHW  (Office  of  Missile  Electronic  Warfare).  White  Sands  Missile 
Range,  New  Mexico,  were  selected  for  study.  The  variability  of  the  measurements  taken 
with  these  equipments,  subject  to  the  procedures  routine  to  their  use.  is  the  topic  of  this 
report.  It  is  anticipated  that  subsequent  phases  of  the  uncertainty  analysis  will  include 
aircraft  signature  data  collected  by  the  Pomona  Division  of  General  Dynamics  Corporation, 
which  is  a  third  major  measurement  facility  using  the  same  type  of  interferometer. 


OBJECTIVES 

The  goal  of  this  study  is  to  determine  quantitatively  the  variability  of  data  mease  ed 
with  an  interferometer  system  and  associated  routine  procedures,  independent  of  any 
effects  introduced  by  atmosphere,  background,  or  parameters  characteristic  of  the  source 
of  radiation.  The  factors  contributing  to  the  variability  discovered  will  be  identified  based 
on  a  physical  understanding  of  the  measurement  system  and  process.  Finally,  recommen¬ 
dations  are  listed  for  analyzing  and  improving  the  measurement  process  and  for  interpreting 
data  collected  from  such  measurements. 


APPROACH 

The  data  base  for  this  study  was  collected  under  conditions  chosen  to  minimize  the 
effects  of  atmosphere,  background,  and  radiation  source.  A  calibrated  blackbodv  source 
was  measured  to  provide  a  target  that  could  be  theoretically  described  for  later  comparison 
to  the  measured  result.  The  measurements  were  made  at  night  to  minimize  background 
contributions.  Finally,  an  integrated  value  of  spectral  radiant  intensity  from  3.5  microm¬ 
eters  (/on)  to  4.0  /on  wavelength  was  chosen  for  data  analysis  to  take  advantage  of  the 
atmospheric  window  to  IR  radiation  in  that  wavelength  region,  thus  minimizing  atmos¬ 
pheric  effects. 

Two  Michelson  interferometer  measurement  systems,  including  dedicated  computer 
equipment  for  performing  a  Fast  Fourier  Transform,  were  used  to  measure  blackbodv 
sources  over  a  variety  of  source  temperatures,  source  apertures,  and  distances  to  tareet. 
The  possible  combinations  of  these  three  variables  generated  a  matrix  of  data  cells  over 
three  parameters.  Whenever  sufficient  data  were  available,  thirty  measurements  of  integrated 
spectral  radiant  intensity  from  3.5/im  to  4.0/mi  were  obtained  for  each  data  cell. 

A  statistical  study  of  this  body  of  data  was  performed  to  determine  the  data  spread 
within  each  data  cell.  Numerical  analysis  techniques  were  then  applied  to  try  to  determine 
the  specific  causes  of  variability. 

The  Michelson  Interferometer 


The  Michelson  interferometer  is  a  rapid-scan,  high  spectral  resolution  device  which  is 
capable  of  observing  a  broad  spectrum  of  IR  radiation  from  an  emitting  source.  The 
interferogram  which  results  from  one  scan  of  the  device  is  converted  to  a  spectral  format 
by  the  mathematical  technique  of  the  Fourier  transform,  accomplished  by  dedicated  com¬ 
puter  equipment  which  must  be  considered  as  part  of  the  measurement  system. 
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To  properly  determine  that  contribution  to  variability  which  is  introduced  separately 
by  each  of  such  factors  as  the  width  of  the  “windows”  through  which  the  interferogram 
is  digitized  by  the  computer,  ambiguity  in  mathematical  approximations  internal  to  the 
computer  process,  and  so  forth,  would  require  an  expensive  and  expert  study  beyond  the 
scope  of  the  present  effort.  The  total  effect  of  all  these  factors  may  by  included  in  the 
assessment  of  the  variability  of  an  interferometer  measurement  sample  for  the  purposes 
of  this  study. 

Summary  of  Data  Collected 

Measurement  data  for  this  study  were  collected  at  facilities  operated  by  NWC  and 
OMHW.  The  data  were  reported  in  graphical  and  digital  form,  giving  apparent  radiant 
intensity  as  a  function  of  wavelength.  For  each  graph,  the  spectral  radiant  intensity  was 
integrated  from  3.5pm  to  4.0pm  to  give  a  single  radiant  intensity  value  for  the  interval.  This 
integrated  value,  effective  apparent  radiant  intensity,  J,  which  may  be  described  as 

C  4.0  pm 

•*eff  app  “1  V‘X 

J  3.5  pm 

where  is  apparer*  spectral  radiant  intensity,  was  reported  by  both  facilities.  For  every 
combination  of  temperature,  aperture,  and  distance  values,  thirty  samples  of  this  basic 
data  item  were  obtained  whenever  possible.  The  complete  data  matrix  obtained  for  this 
effort  is  listed  in  Table  1 .  Figures  1 ,  2,  and  3  show  samples  of  the  data  supplied  by  the  two 
measurement  facilities. 

Both  measurement  facilities  participating  in  this  study  customarily  report  their  data 
as  apparent  spectral  radiant  intensity.  For  most  users  of  the  data  this  is  a  convenient  format, 
since  the  physical  problem  of  interest  is  the  1R  emission  of  an  aircraft.  What  any  facility 
actually  measures,  however,  is  radiation  incident  on  the  collecting  optics  of  the  measurement 
equipment,  located  at  some  distance  front  the  source  of  emission.  This  irradiance  measure¬ 
ment  is  mathematically  converted  to  radiant  intensity  by  the  facility  for  reporting  purposes. 

The  impact  of  distance  on  the  variability  of  IR  measurements  is  an  important  con¬ 
sideration  in  the  analysis  described  in  this  report.  Therefore,  all  the  data  received  from  NWC 
and  OMHW  were  reconverted  from  radiant  intensity  to  irradiance  using  the  same  values 
for  distance  as  those  facilities  used  for  the  original  conversion.  Therefore,  the  basic  data 
item  for  this  study  is 


f  4.0  pm 

^3.5pm  -  4,0pm  =  \  dX 

J  3.5  pm 

where  is  spectral  irradiance.  There  are  about  thirty  samples  of  this  data  item  in  each 
of  the  45  data  cells.  In  addition,  each  cell  corresponds  to  one  measurement  each  of  the 
three  parameters  -  temperature,  aperture,  and  distance. 
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Table  1.  List  of  Measured  Data  Sets. 


Distance, 

in 


Temperature. 

°K 


Facility:  NWC.  China  Lake 


30.48 

30.48 

30.48 

30.48 

30.48 

30.48 

30.48 

*30.48 

30.48 

30.48 

30.48 

30.48 

60.  % 
60.% 
60.^6 
60% 

60.46 

60.46 


805.2 

805.2 

805.2 

805.2 

444.2 

443.2 

440.2 

440.2 

1 143.2 

1 143.2 
I  154.7 
1 160.7 

806.2 

806.2 

806.2 

806.2 

443.2 

488.2 


Aperture, 
cm  ~ 


5.067 

29.200 

77.070 

126.700 

5.067 

29.200 

77.070 

126.700 


5.067 

29.200 

77.070 

126.700 


5. 06  7 
29.200 
72.450 
101.300 

5.067 
29.200 


60.96 


994.2 


101.300 


60.96 

60.96 


1 140.2 
1  145.7 


5.067 

29.200 


60.96 


1 141.5 


101.300 


*Dita  Mt  hat  only  27  mambarv 
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Tabic  1 .  List  of  Measured  Data  Sets  (Contd.). 


Distance. 

in 

Temperature. 

°, 

Aperture. 

cm" 

152.40 

804.4 

77.070 

152.40 

804.7 

26.700 

152.40 

986.2 

5.067 

152.40 

985.7 

29.200 

152.40 

986.2 

77.070 

152.40 

1126.2 

5.067 

152.40 

1 142.7 

29.200 

152.40 

1143.2 

77.070 

152.40 

1148.2 

126.700 

152.40 

1 148.2 

5.067 

152.40 

1143.2 

29.200 

152.40 

1 142.7 

152.40 

1 126.2 

126.700 

Facility:  OMLW,  White  Sands 


155 

950.26 

0.713 

155 

950.26 

5.067 

*  155 

950.26 

20.270 

155 

950.26 

45.600 

155 

1 155.86 

0.713 

155 

1155.86 

5.067 

155 

11 55.86 

20.270 

*155 

1155.86 

45.600 

155 

1423.76 

20.270 

155 

1423.76 

45.600 

*Oata  HI  hat  only  29  mam  bar*. 
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Figure  I.  Sample  of  China  Lake  Data  in  Graphical  Format. 
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3 . 83434 

9.9596 

3.48389 

9 . 46099 

3 . 84572 

9.52492 

3.49323 

9 . 76748 

3.85717 

9 . 5976 

3.50272 

10.2451 

3 . 86863 

10.6876 

3.51222 

9 . 5338 

3 . 83326 

9.64145 

3.52176 

9.81474 

3.89192 

9.46629 

3.53136 

10.0853 

3 . 9G364 

9.74356 

3.54101 

9.43727 

3.91543 

9.79393 

3.55071 

9.44336 

3 . 9273 

9 . 33596 

3 . 56046 

9  55742 

3  93Q23 

9 . 26304 

3.57027 

9 . 06377 

3.95124 

9 . 7S2S3 

3.58G13 

9. 18507 

3 . 96333 

9.48575 

3 . 59O05 

9.62242 

3 . 97548 

9 . 54483 

3 . 600O2 

9.7106 

3 . 96772 

9.74743 

3 . 61005 

9.79616 

4 . 000O3 

9.27392 

3.62013 

9 . 85363 

4.01241 

9.3051 

3 . 63027 

9.66141 

4 . 02437 

9.6642 

3.64047 

9 . 67863 

4.03741 

9.54123 

3 . 65073 

9.91 1 

4 . 05003 

8.99974 

3.66104 

9 . 47626 

3.67141 

9 . 40357 

3.68184 

9.67  o  5 

3 . 69233 

10.1045 

3 . 70288 

9 . 32509 

3.71349 

9 . 74622 

3.72416 

9 . 9205 

3 . 73489 

9.51038 

3 . 74369 

9.56684 

3 . 75654 

9.44898 

3 . 76747 

9 . 3907’ 

3 . 77345 

9.7245 

3 . 78S5 

9.42853 

3.eessi 

9.8651 

3.81179 

9 . 66869 

3.82333 

9.3728 

Figure  2.  Sample  of  China  Lake  Data  in  Digital  Format. 
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THE  INFRARED  MEASUREMENT  UNCERTAINTY  PROBLEM 


THEORY  OF  MEASUREMENT  ERROR 
The  Meaning  of  Error 

A  physical  quantity  may  he  assumed  to  have  a  unique  value*,  unknown  to  man,  which 
scientists  attempt  to  determine  as  closely  as  possible  by  performing  experimental  measure¬ 
ments.  This  unique  but  unknown  value  is  called  the  true  value.  The  value  of  the  physical 
quantity  which  is  determined  by  experimental  measurement  is  called  the  measured  value. 
The  increment  by  which  the  measured  value  differs  from  the  true  value  is  described  as 
measurement  error.  These  concepts  are  illustrated  for  a  single-valued  quantity  which  can 
be  expressed  with  a  real-number  value  in  Figure  4.  Only  the  measured  value  of  the  physical 
quantity  is  known.  Since  the  true  value  is  unknown,  the  magnitude  of  the  measurement 
error  is  also  unknown.  The  goal  of  this  section  is  to  discuss  all  those  details  which  can  be 
learned  about  the  measurement  error  based  on  (1)  a  knowledge  about  the  measurement 
process,  and  (2)  an  examination  of  the  empirical  data  resulting  from  the  measurement 
process. 


I-' —  Measurement  Error  — J 


x— 

Origin 


True  Value  Measured  Value 

Figure  4.  Measurement  Error. 


Axis  of  Real  Numbers 


Three  Kinds  of  Measurement  Error 


In  general,  measurement  error  is  not  the  result  of  just  one  contributing  factor.  How¬ 
ever,  all  of  the  measurement  error  effects  which  derive  from  the  physical  configuration  of 
a  measurement  system  or  from  the  procedures  for  use  of  the  system  can  be  grouped  into 
three  categories.  These  are: 

1 .  Mistakes,  or  blunders 

2.  Constant,  or  systematic,  errors 

3.  Accidental,  or  random,  errors 

The  errors  in  the  second  category  arc  often  called  bias. 

•Pxduding  the  consideration*  of  quantum  physics. 
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As  an  illustration  of  these  three  error  categories,  consider  the  case  of  a  man  who 
measures  the  length  of  several  rods,  each  of  which  has  a  specific  function  in  some  equipment. 
Suppose  that  he  performs  the  measurements  on  a  warm  day  by  using  a  steel  measuring  tape. 
Due  to  the  increased  temperature  of  the  day,  the  steel  tape  has  expanded  from  the  length 
it  was  when  the  markings  were  imprinted  on  it.  Also,  the  tape  has  a  loop  on  the  zero  end 
for  convenience  in  making  hand-held  measurements.  The  tape  is  marked  off  with  centi¬ 
meters  as  the  smallest  scale  division. 

If  the  man  makes  a  measurement  showing  that  a  rod  is  24.3  centimeters  in  length,  and 
writes  down  the  number  27.3  in  his  data  book  due  to  inattention,  this  is  a  blunder.  Also,  he 
might  make  the  mistake  of  recording  the  correct  number,  24.3,  but  attributing  this  number 
in  his  notes  to  the  wrong  rod. 

The  hasp  by  which  the  holding  loop  is  attached  to  the  zero  end  of  the  tape  may  cause 
the  end  of  the  tape  always  to  be  improperly  aligned  with  the  end  of  each  measured  rod. 
causing  all  the  measurements  to  be  too  small  by  the  constant  width  of  the  hasp.  Also, 
since  the  tape  has  expanded,  even  if  it  were  properly  aligned,  all  the  measurements  would 
suffer  the  systematic  error  of  being  a  fraction  of  what  they  should  be  namely,  the  ratio 
of  the  original  tape  length  to  the  new  tape  length.  If  the  man,  in  estimating  the  distance 
between  the  smallest  scale  divisions  marked  on  the  tape,  always  tends  to  read  low.  this 
will  also  introduce  a  systematic  error  into  the  measurements. 

Finally,  there  are  many  measurements  which  are  equally  likely  to  be  high  or  low. 
The  distribution  of  estimates  between  smallest  scale  divisions  over  and  above  the  personal 
equation  of  the  man  making  the  measurements  is  random  in  nature.  So  might  be  the 
alignment  of  the  zero  end  of  the  tape,  after  the  constant  effect  of  the  hasp  has  been  ac¬ 
counted  for. 


The  maximum  possible  error  which  can  occur  in  a  measurement  excluding  blunders 
is  that  error  which  results  when  the  effects  of  all  errors  are  as  additive  as  possible  given 
the  constraints  of  the  measurement  process.  This  would  occur  in  the  caseot  the  man  measur¬ 
ing  rods  when: 

( 1 )  The  hasp  is  included  in  the  measurement. 

(2)  The  tape  is  stretched. 

<3)  The  man  consistently  estimates  low. 

(4)  A  low  reading  in  the  distribution  of  the  man’s  estimates  occurs. 

(5)  The  rod  tends  to  overhang  the  hasp  in  this  particular  instance  which  has  been 
selected  from  the  distribution  of  possible  alignments. 


If  all  these  effects  coincide,  the  maximum  error  on  the  low  side  of  true  value  will  be  achieved. 


JTCG/AS-79-C-003 


The  maximum  possible  error  on  the  low  side  of  true  value  does  not  necessarily  equal 
that  on  the  high  side.  In  the  example  just  given,  if  the  personal  equation  of  the  man  always 
caused  high  estimates,  this  would  somewhat  counteract  the  other  effects.  Also,  the  presence 
of  the  hasp  will  always  lower  the  measured  value  from  the  true  value,  whereas  the  tape 
may  either  expand  or  contract  with  temperature. 

Systematic  and  random  error  are  shown  in  Figure  5  as  contributing  in  additive  fashion. 
A  probabilistic  distribution  of  random  error  is  indicated  in  the  figure. 


«*■ 


Measurement  Error 


Magnitude  of 
Systematic  Error 


Magnitude  of 
Random  Error 


Figure  S.  Additive  Systematic  and  Random  Error. 


Mistakes,  or  blunders,  cannot  be  predicted.  They  can  be  controlled  by  good  procedure 
and  careful  attention  to  detail  during  data  collection.  Large  mistakes  may  also  be  discovered 
by  inspection  of  the  data  and  eliminated  by  recomputation  of  results  in  some  cases. 

Systematic  errors  affect  all  measurements  in  the  same  way.  They  are  usually  due  to 
the  adjustment  or  calibration  of  instrumentation,  inherent  physical  properties  of  the  measure¬ 
ment  system,  or  the  personal  equations  of  members  of  the  measurement  team.  These  errors 
can  sometimes  be  detected  by  mathematical  analysis. 

Random  errors  are  usually  dealt  with  in  measurement  programs  by  taking  several 
data  items  and  reporting  the  average  value  of  the  sample  data  set.  This  method  assumes  that 
the  probabilistic  distribution  of  random  error  for  the  measurement  process  in  question  has 
finite  mean  and  variance. 
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Inspection  of  Measurement  Data 
by  Graphing 

The  effects  of  the  three  types  of  measurement  error  on  data  can  he  illustrated  graphi¬ 
cally  as  shown  in  Figure  6.  If  the  true  value  of  a  physical  quantity  could  be  known,  a 
graph  of  measured  value  as  a  function  of  true  value  could  be  plotted.  For  perfect  measure¬ 
ments  this  would  result  in  the  straight  line  at  45  degrees  inclination  shown  in  the  Figure. 
The  curve  resulting  from  imperfect  measurements  is  indicated  by  the  dashed  line.  The 
dashed  line  may  be  displaced  from  the  solid  line  by  a  constant  amount  everywhere  along 
its  length  (constant  error)  or  it  may  be  displaced  by  a  systematically  changing  increment 
(systematic  error)  or  by  a  combination  of  these  effects. 

The  data  points  are  indicated  by  circles.  A  large  blunder  can  usually  be  detected  bv 
inspection  for  a  continuous  function  such  as  the  one  depicted  here.  The  dotted  lines  on 
either  side  of  the  dashed  line  represent  the  spread  of  random  variation  in  the  data. 


Measured 

Value 


Random 

Error 


Figure  6.  Graphical  Inspection  of  Measurement  Data. 
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This  analysis  technique  can  be  applied  to  the  data  collected  for  this  study  if  atmos¬ 
pheric  effects  and  system  responsivity  are  included  as  part  of  systematic  error.  The  true 
value  of  the  physical  quantity  estimated  by  each  data  sample  is  not  known.  However, 
Planck’s  Law  can  be  used  to  produce  a  calculated  value  of  integrated  spectral  irradiance 
associated  with  each  sample  set  of  data.  The  comparison  of  measured  irradiance  with 
irradiance  values  calculated  from  Planck’s  Law  is  discussed  in  the  section  of  the  report 
entitled  Significant  Systematic  Differences. 

Precision  and  Accuracy 

The  term  precision  describes  the  repeatability  of  an  experiment,  or  the  success  with 
which  the  scientist  can  closely  reproduce  the  same  measured  value  throughout  several 
trials.  The  spread  of  the  probabilistic  distribution  of  random  error  is  an  indication  of  pre¬ 
cision.  The  magnitude  of  the  displacement  of  the  mean  measured  value  over  several  trials 
from  the  true  value  relates  to  accuracy,  which  describes  the  ability  of  the  scientist  to  achieve 
a  measured  value  close  to  the  true  value.  Precision  is  improved  by  reducing  the  spread  of 
random  error;  accuracy  is  improved  by  decreasing  systematic  error. 

These  concepts  can  be  described  in  terms  of  the  experiment  previously  examined  in 
which  the  lengths  of  several  rods  were  to  be  measured.  If  the  man  were  to  make  some 
intelligent  accounting  for  the  width  of  the  hasp  on  his  measuring  tape,  the  accuracy  of 
his  measurements  would  be  improved.  If  he  were  to  obtain  a  tape  marked  off  with  milli¬ 
meters  as  the  smallest  scale  division,  his  data  would  be  more  precise. 

Three  Ways  of  Expressing 
Deterministic  Error 

Let  X  represent  the  true  value  of  a  physical  quantity. 

Let  X  +  AX  be  the  measured  value. 

Then 


AX  -  (X  +  AX)  -  X  is  called  the  absolute  error 

AX/X  is  called  the  relative  error 

100  (AX/X)  is  called  the  percentage  error 

These  are  the  three  most  commonly  used  ways  of  presenting  deterministic  estimates  of 
error. 

Probabilistic  Characterization 
of  Accidental  Error 

If  X  is  the  true  value  of  a  physical  quantity,  then  Xj  =  (X  +  AXf)  is  the  measured 
value  of  the  ith  measurement,  where  AXf  is  a  stochastic  component  consisting  of  the  sum 
of  the  three  basic  error  types: 

AXj  -  AXj  blunder  +  AXj systematic  +  ^i.random 
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It'  the  contributions  of  blunder  have  been  eliminated  by  careful  inspection  of  the 
data,  and  if  the  third  error  term  is  truly  random,  with  mean  zero,  then  it  should  be  possible 
to  isolate  the  systematic  error  term  by  taking  several  measurements  and  averaging  them  to 
“zero  out”  the  third  term.  That  is.  for  N  measurements, 

N  N  N 

S  S  ^^i.systematic  2-  AX(- random 

M  _  i-l  i=l 

N  N  N 


where 


N 

^  ^i, random 
i=l 

N 


0  as  N  -*  « 


I  or  many  measurement  processes  it  is  reasonable  to  describe  the  random  component 
of  error  AX|  |n(|om  as  having  normal  (Gaussian)  distribution*.  The  density  function  of 
a  typical  hypothetical  infinite  population  of  random  error  is  illustrated  in  Figure  5.  Several 
characteristics  which  may  be  defined  for  a  sample  of  N  data  items  Xj  drawn  from  such  a 
population  arc  delined  in  the  Appendix,  Statistical  Definitions.  These  include  sample  mean 
and  sample  standard  deviation. 

As  shown  in  Figure  5,  the  sample  mean  is  an  indication  of  systematic  error  (or  accu¬ 
racy):  the  standard  deviation  is  descriptive  of  random  error  (or  precision). 

Algebraic  Expressions  for  Error 

For  any  physical  quantity  which  can  be  expressed  as  a  deterministic  function  of 
several  measurable  parameters,  it  is  possible  to  describe  the  exact  error  in  the  computed 
value  of  that  quantity  as  a  function  of  the  exact  values  of  errors  in  the  measured  parameters. 
The  following  basic  rules  apply: 

I  ora  sum  |A  +  B|,  the  error  is 

I  (A  +  AA)  +  (B  +  AB)  |  -  | A  +  B |  =  AA  +  AB 

l  or  a  product  |  A  •  B|  the  error  is 

I  (A  +  AA)  •  (B  +  AB)  |  -  |A  •  B)  =  BAA  +  AAB  +  AAAB 

For  a  square  |  A“|  the  error  is 

I  (A  +  AA)2  |  -  | A“|  =  2  AAA  +  AA2 

*The  term  "normal"  for  this  widely  used  distribution  was  introduced  by  k.  Pearson  and  is  the  term  usually  used 
in  applied  statistics  today.  The  same  distribution  has  hern  referred  to  as  the  second  law  of  I  aplace.  the  I  aplace  distri¬ 
bution.  the  Gauss  and  the  l.aplace-Gauss  distribution. 
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Similar  considerations  apply  lor  all  powers,  differences,  quotients,  logarithms,  ex¬ 
ponentials,  etc.,  but  the  expressions  in  some  cases  are  more  complex.  In  the  discussion 
of  random  errors,  where  AA  and  AB  are  random  variables,  the  resulting  probabilities,  to¬ 
gether  with  the  possibilities  that  the  true  values  A  and  B  may  be  either  positive  or  negative, 
lead  to  a  tedious  computation  of  distribution  functions  and  proliferation  of  cases  even 
for  sums,  products,  and  squares.  The  exact  analysis  relates  to  the  determination  of  the 
distribution  of  products,  sums,  and  more  arbitrary  functions  of  possibly  dependent  random 
variables.  The  analysis  required  in  the  following  does  not  require  this  exact  procedure 
for  small  error  analysis  (to  the  order  of  A,  ignoring  terms  of  order  A"  and  higher). 


MEASUREMENT  UNCERTAINTY 

Causes  of  Measurement  Uncertainty 
for  One  Measurement  System 


Whenever  a  radiating  source  and  an  IR  measurement  system  are  positioned  for  the 
purpose  of  collecting  data  describing  the  source  emissions,  a  unique  set  of  physical  con¬ 
ditions  prevails  at  any  instant  which  determines  without  ambiguity  the  irradiance  at  the 
collecting  optics  of  the  instrument.  This  value  is  referred  to  throughout  this  report  as  the 
true  value  of  irradiance.  Certainly  the  physical  conditions  must  always  include  background 
and  atmospheric  effects.  Eor  a  blackbody  source,  they  also  include  as  a  minimum  the 
source  emissivity,  temperature,  and  aperture,  and  the  separation  distance  of  the  source 
from  the  measurement  system,  l-'or  a  more  complex  source  such  as  an  operating  aircraft, 
many  other  parameters  must  be  identified,  including  those  describing  engine  conditions 
and  plume  gas  dynamics. 

If  a  sample  of  several  measurements  is  taken  corresponding  to  this  unique  set  of 
physical  conditions,  the  items  in  the  sample  will  be  probabilistically  distributed.  A  sample 
mean  and  variance  can  be  calculated.  Then  brackets  about  the  sample  mean  can  be  specified 
which  can  be  said  with  some  degree  of  confidence  to  contain  the  true  but  unknown  mean 
of  the  infinite  population  of  possible  measurements  for  this  unique  experimental  configu¬ 
ration.  If  there  is  no  bias  in  the  measurement  process  and  the  true  value  does  not  change 
over  the  measurement  period,  then  the  mean  of  the  infinite  population  of  possible  measure¬ 
ments  will  be  the  true  value  of  the  irradiance  at  the  collecting  optics.  As  the  sample  size 
drawn  from  a  normally  distributed  population  becomes  larger,  the  brackets  about  the 
sample  mean  which  are  said  to  contain  the  population  mean  become  tighter  for  any  given 
level  of  confidence  (i.e..  the  length  of  the  confidence  interval  decreases  inversely  as  the 
square  root  of  the  sample  size).  If  there  is  bias  in  the  measurement  process,  the  population 
mean  will  be  displaced  from  the  true  value  of  irradiance  by  the  amount  of  the  bias. 

The  probabilistic  distribution  of  individual  measurements  describes  one  source  of 
measurement  uncertainty.  The  same  equipment  measuring  the  same  unique  physical  con¬ 
ditions  will  get  different  results  from  sample  to  sample.  This  concept  is  illustrated  in 
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Figure  7.  Means  of  Several  Samples  Drawn  From  a  Population  Having  Mean  /ijj. 


If  there  is  any  bias  in  the  measurement  process  which  can  change  during  the  course 
of  an  experiment,  then  the  same  physical  conditions  measured  under  two  different  con¬ 
ditions  of  bias  will  yield  results  that  differ  by  the  additive  effects  of  bias,  as  well  as  by 
sample-to-sample  variation.  An  example  of  this  type  of  bias  is  the  amount  by  which  the 
system  response  factor  programmed  into  the  measurement  process  during  calibration 
differs  from  the  true  but  unknown  value  of  response.  This  concept  of  changing  bias  is 
illustrated  in  Figure  8.  The  true  value  of  response  is  denoted  by  R.  On  the  first  calibration 
a  value  of  response  (R  +  AR  | )  is  measured.  On  the  second  calibration  a  value  of  response 
(R  +  AR-»)  is  measured.  The  bias  for  each  measurement  sequence  is  a  function  of  the 
error  in  measured  response.  Within  the  range  of  bias  variation,  the  same  equipment  mea¬ 
suring  the  same  unique  physical  conditions  will  get  different  results  from  bias  condition 
to  bias  condition.  In  fact,  all  calibrated  systems  have  some  bias,  since  the  one-shot  system 
response  measurement  is  itself  selected  from  a  probabilistic  distribution  of  possible  mea¬ 
sured  response  values. 
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Sequence  #1 


Population 

Mean 


True 

Value 


K 


Variation  in  Data 
Due  to  Recalibration 


Figure  8.  Effect  of  Changing  Bias. 


The  two  causes  of  data  uncertainty  just  described  occur  even  though  a  unique  set 
of  physical  conditions  prevails.  In  addition,  there  is  the  problem  of  reproducing  the  unique 
set  of  physical  conditions.  Suppose  a  particular  source  of  IR  emissions  is  measured  by  a 
given  system  on  one  occasion,  and  remeasured  later  by  the  same  system.  With  what  close¬ 
ness  can  the  background  or  atmospheric  factors  be  measured  in  each  case?  With  what  accu¬ 
racy  can  the  same  distance  to  target,  the  same  geometric  orientation  of  the  source,  the  same 
blackbody  temperature  or  the  same  engine  operating  conditions  be  measured  and  repro¬ 
duced?  In  short,  what  is  the  variability  in  the  process  of  attempting  to  reproduce  the  exact 
same  value  of  irradiance  at  the  collecting  optics?  This  is  the  third  cause  of  data  uncertainty 
among  measurements  made  by  the  same  system  the  reproducibility  with  which  the  true 
value  of  irradiance  for  one  experiment  can  be  duplicated  on  a  different  occasion  even  if 
that  occasion  occurs  only  moments  later. 
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Understanding  Uncertainty 
for  One  Measurement  System 

A  pictorial  representation  of  the  three  causes  of  uncertainty  among  data  describing 
the  same  source  of  emissions  and  measured  by  the  same  equipment  is  given  in  Figure  9. 
The  arc  BC  indicates  the  range  of  reproducibility  due  to  human  inability  to  measure  accu¬ 
rately  the  physical  conditions,  or  parameters,  which  determine  a  unique  experimental  con¬ 
figuration.  The  pointer  at  the  top  of  the  figure  is  pivoted  at  a  fixed  point  A.  which  repre¬ 
sents  the  irradiance  at  the  collecting  optics  that  is  assumed  to  exist.  The  tip  of  this  pointer 
may  select  any  value  1)  from  the  arc  BC,  where  I)  represents  the  true  irradiance  that  does 
in  fact  exist. 

The  tip  of  the  first  pointer  is  the  pivot  point  for  the  second  pointer,  whose  range  of 
variation  along  the  arc  FF  represents  the  probabilistic  introduction  of  bias  into  the  measure¬ 
ment  process.  The  point  I)  from  which  the  second  pointer  is  pivoted  may  lie  anywhere 
along  the  arc  BC,  but  once  its  location  is  determined  by  a  unique  experimental  configu¬ 
ration,  the  tip  of  the  second  pointer  is  constrained  to  vary  over  an  arc  FF  symmetric  about 
a  midpoint  directly  below  I).  (This  assumes  that  each  bias  value  is  a  sample  of  one  drawn 
from  the  distribution  of  possible  values  for  a  random  bias  variable.) 

The  tip  of  the  second  pointer,  denoted  by  ('.,  serves  as  a  pivot  for  the  third  pointer 
at  the  bottom  of  the  figure.  Once  a  bias  value  has  been  determined,  by  calibration  error  and 
perhaps  by  other  system  characteristics,  the  point  G  is  fixed.  The  tip  of  the  third  pointer 
may  travel  along  arc  HJ.  symmetric  about  a  midpoint  directly  below  Cl,  where  the  arc  HJ 
represents  sample-to-sample  variation  in  data  measured  for  one  unique  configuration.  The 
tip  of  the  third  pointer  K  represents  the  value  of  irradiance  measured  by  the  system. 

This  representation  is  not  a  completely  analogous  one.  The  range  of  the  Gaussian, 
or  normal,  distribution  with  which  random  error  has  been  characterized  is  infinite:  all 
three  arcs  are  shown  as  finite  in  length.  Moreover,  the  points  near  the  center  of  the  arcs 
have  a  higher  probability  of  being  selected  than  the  points  near  the  ends.  However,  this 
figure  is  useful  for  visualizing  the  way  in  which  the  three  causes  of  uncertainty  in  measure¬ 
ment  data  from  the  same  measurement  system  may  supplement  or  counteract  each  other, 
and  how  they  all  contribute  to  the  total  uncertainty  of  the  measurement  data  from  an 
experiment. 

The  maximum  possible  error  occurs  when  all  three  pointers  are  extended  to  the  left 
ends  of  their  possible  arcs,  or  when  all  three  pointers  are  extended  to  the  right  ends  of  their 
possible  arcs.  The  maximum  possible  difference  between  two  measurements  of  the  same 
perceived  physical  situation  made  with  the  same  measurement  system  is  represented  by  the 
difference  between  the  two  maximum  possible  error  configurations  just  described. 

The  range  of  the  tip  of  the  second  pointer  is  descriptive  of  the  accuracy  of  the  measure¬ 
ment  system.  The  range  of  the  tip  of  the  third  pointer  relates  to  measurement  system  pre¬ 
cision.  The  first  pointer  represents  the  reproducibility  of  the  experiment,  or  the  ability 
to  achieve  the  same  value  of  irradiance  at  the  collecting  optics  of  the  measurement  system. 
It  has  no  relationship  to  the  measurement  system. 
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The  Dellning  Equations 

for  an  Experimental  Configuration 


An  experimental  configuration  may  he  described  by  writing  an  equation  or  sequence 
of  equations  to  define  measurement  system  output  as  a  function  of  all  the  physical  para¬ 
meters  which  determine  the  configuration.  When  this  is  done  for  the  blackbody  experiment 
discussed  in  this  report,  the  sequence  of  equations  gives  (1)  irradiance  at  the  collecting 
optics  of  the  measurement  system  as  a  function  of  the  physical  parameters  which  determine 
the  configuration,  and  (2)  system  output  as  a  function  of  irradiance  at  the  collecting  optics 
of  the  measurement  system. 


The  first  equation  for  the  blackbody  experiment  has  the  form 

»4.0  pm 


(I) 


J4.u  pm 

\ 

3.5  pm 


W(X,T)  •  r(X,D)  dX 


where 

II  is  the  irradiance  at  the  collecting  optics  of  the  measurement  system 
A  is  the  presented  area  of  the  aperture  of  the  source  of  emissions 
e  is  the  emissivity  of  the  source 

I)  is  the  distance  from  the  source  to  the  entrance  aperture  of  the  measurement  system 
collecting  optics 

W  is  the  spectral  radiant  omittance  of  the  source 
r  is  the  spectral  transmittance  of  the  atmosphere 

The  spectral  irradiance  at  the  collecting  optics  of  the  measurement  system  is  defined  as 

H(X)  =  W(X.T)r(X,D) 

7rl)2 

The  voltage  signal  produced  by  the  irradiance  at  the  collecting  optics  is 
V(X)  =  l!(X)R(X) 

where  R(X)  is  the  spectral  response  of  the  optical  system,  detector,  and  system  electronics. 
Finally  a  transfer  function  F  can  be  said  to  operate  on  the  system  output  V(X),  where  the 
operator  F  represents  the  dedicated  computer  processing  including  calibration  effects, 
the  Fast  Fourier  Transform,  and  the  integration  from  3.5  pm  to  4.0  pm.  The  second  equa¬ 
tion  for  the  blackbody  experiment  has  the  form 

^  ^3,5  pm  -  4.0  pm  “  I" I H(X)  •  R(X)| 

2l 


i 

1 
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The  qualities  of  accuracy  and  precision  for  the  experiment  relate  to  the  processes 
described  by  the  second  equation.  Therefore,  the  second  equation  describes  that  portion 
of  the  configuration,  namelv  the  measurement  system,  which  introduces  the  variability 
represented  by  the  second  and  third  pointers  in  Figure  9.  That  is,  calibration  bias  and  any 
other  system  bias  are  introduced  by  the  operator  F.  The  very  process  of  measuring 
starting  with  irradiance  H(X)  at  the  collecting  optics  and  ending  with  values  for  H  printed 
out  from  the  computer  terminal  -  introduces  sample-to-sample  variability. 

The  reproducibility  of  the  experiment  is  determined  by  the  first  equation.  The  ambi¬ 
guity  in  the  true  value  of  irradiance  at  the  collecting  optics,  given  what  appear  to  be  success¬ 
fully  reproduced  values  of  the  defining  parameters,  can  be  expressed  in  terms  of  the  measure¬ 
ment  variances  of  the  defining  parameters. 

The  next  three  sections  of  this  chapter  discuss  the  analysis  of  the  three  causes  of  data 
uncertainty  for  the  same  source  used  with  the  same  measurement  system. 

Quantifying  Sample-to-Sample 
Variability 

The  variability  associated  with  the  probabilistic  distribution  of  measurements,  which 
is  a  fact  of  nature  that  cannot  be  overcome,  can  be  studied  and  quantified  for  any  measure¬ 
ment  system  so  that  its  contribution  to  the  measurement  process  is  understood.  Statistical 
and  numerical  analysis  can  be  conducted  to  determine  the  functional  relationship  between 
measurement  variance  and  the  magnitude  of  the  measured  parameter.  This  has  been  at¬ 
tempted  for  the  two  systems  studied  in  this  report.  Several  mathematical  functions  were 
fitted  to  the  data  in  an  attempt  to  discover  the  appropriate  formula  for 

o  =  f  (/j|^,B) 


where 

a  is  the  standard  deviation  of  the  measurement  population  corresponding  to  a  given 
set  of  physical  conditions 

is  the  mean  of  the  measurement  population,  and,  in  the  absence  of  bias,  is  the  true 
value  of  irradiance  at  the  collecting  optics  of  the  measurement  system 

B  is  any  bias  which  displaces  pjj  from  the  true  value  of  irradiance 

This  analysis  of  the  measurement  data  from  China  Lake  and  White  Sands  is  described  under 
Analysis  of  the  Measured  Values. 

Quantifying  Reproducibility 

The  physical  parameters  which  determine  an  experimental  configuration  may  all 
be  measured  when  an  experiment  is  performed.  If  an  attempt  is  made  to  repeat  the  experi¬ 
ment,  the  physical  parameters  will  all  be  measured  a  second  time,  and  adjusted  until  they 
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seem  to  have  the  same  values  as  they  did  in  the  first  case.  (For  the  blackbody  experiment 
the  defining  parameters  listed  were  temperature,  aperture,  distance,  emissivity,  anil  atmos¬ 
pheric  transmission. )  On  both  the  first  and  second  occasions,  every  last  one  ol  these  measured 
parameter  values  is,  by  the  very  nature  of  things,  in  error  to  some  extent.  On  each  occasion, 
the  measured  value  of  irradiance  is  recorded  as  being  the  result  of  all  the  erroneous  values  of 
the  physical  conditions. 

The  true  value  of  irradiance  Hy  at  the  collecting  optics  of  a  measurement  system  can 
be  expressed  algebraically  in  terms  of  the  defining  equation  for  an  experiment.  For  the 
blackbody  experiment 

H  =  f(T,A.D.e.r) 

A  calculated  value  of  irradiance  Up-  can  be  found  by  inserting  the  measured  parameter 
values  in  the  defining  equation,  if  AT,  AA,  AD.  Ae,  and  At  represent  small,  random  errors 
in  the  measured  values  of  the  defining  parameters  for  the  blackbody  experiment,  the  equa¬ 
tion  for  the  calculated  value  of  irradiance  has  the  form 

H(,  =  l'(T  +  AT, A  +  AA,  D  +  AD.c  +  A e,r  +  At) 


=  WA1 


OF 

OT 


AA  + 


+  0(A  ) 


For  each  defining  parameter  (say  T),  the  measured  value  (say  T  +  AT)  is  a  sample  of  one 
drawn  from  an  infinite  population  of  possible  measured  values.  If  no  bias  exists  in  the 
process  of  parameter  measurement  so  that  F(AT)  =  K(AA)  =  .  .  .  =  O.  then  the  true  but 
unknown  mean  of  the  infinite  population  of  possible  measured  parameter  values  is  the  true 
value  of  the  parameter.*  If  the  measured  values  of  all  the  parameters  are  statistically  inde¬ 
pendent,  random  variables  having  the  true  parameter  values  as  their  means,  the  variability  of 
Hy  about  Hy  can  be  described  in  terms  of  the  parameter  variances.  Specifically,  for  small 
AT,  AA,  and  so  on,  if  Oy,  o^,  Opj,  of,  and  oT  are  the  standard  deviations  of  the  measure¬ 
ment  population  for  T,  A,  D,  e.  and  r,  then  Up-  is  a  random  variable  about  mean  Hy  + 
CM  AT2 )  and 


H 


( 


HF 

3T 


3F 

3A 


Z-\o  2  + 


+  0(A2 1 


If  Hy  is  treated  as  a  constant,  then  the  standard  deviation  of  the  difference  (Up-  -  Hy) 
is  exactly  the  same  as  the  standard  deviation  of  Up-: 


<"(■  *  'V 


•This  statement  ignores  considerations  relating  to  the  limits  of  the  theoretical  and  mathematical  description  of 
the  physical  laws  involved;  i.c.,  it  is  assumed  that  the  basic  formulas  ate  coticct  over  any  range  of  the  parameters  involved. 
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Conversely,  this  is  also  the  standard  deviation  of  Hy  about  Hy.  which  is  the  concent  illus¬ 
trated  by  the  top  pointer  in  Figure  9.  The  equation  for  is  described  under 

Reproducibility  of  the  Experiment  for  the  blackbody  experiment. 

If  there  is  bias  in  the  process  of  parameter  measurement,  then  the  true  value  of  irradi- 
ance  Hy  is  not  the  mean  of  the  distribution  of  possible  calculated  values  Hy  derived  from 
measured  parameter  values.  Instead,  the  mean  of  the  Hy  distribution  is  displaced  from  Ily 
due  to  the  bias.  The  standard  deviation  of  the  Hy  distribution,  and  therefore  of  the 
(Hy  -  Hy)  distribution,  is  unchanged.  Therefore  the  reproducibility  of  data  from  the  same 
measurement  system  is  unaffected.  (This  source  of  bias  must  be  characterized  if  compari¬ 
sons  are  to  be  made  between  two  measurement  systems.) 

Detecting  Bias 

Any  bias  which  is  introduced  by  the  measurement  system  will  be  reflected  in  the 
measured  value  of  irradiance.*  That  is.  the  mean  of  the  infinite  population  of  possible 
measurements  will  not  be  the  direct  result  Hy  of  the  true  values  of  the  defining  parameters, 
but  will  be  the  result  of  those  conditions  displaced  by  some  additional  systematic  effect. 
The  bias  will  not  affect  the  calculated  value  of  irradiance.  Therefore  an  examination  of  the 
pattern  of  values  of  the  discrepancy  (H^  -  lly)  may  reveal  clues  about  system  bias,  where 
Hjyj  is  the  mean  of  a  sample  of  measurements. 

First  it  must  be  determined  that  a  significant  discrepancy  exists.  By  this  is  meant 
a  persistent  difference  between  HM  and  Hy  greater  than  that  reasonably  expected  due  to 
chance  alone,  where  chance  deviations  are  of  the  order  of  differences  between  successive 
measurement  values.  This  can  be  done  by  using  a  statistical  tool  called  the  t-statistic  to 
test  the  hypothesis  that  Hy  is  the  mean  of  the  11^  distribution.  (This  hypothesis  is  unlikely 
to  be  true  for  the  blackbody  experiment  if  only  because  of  the  errors  AT,  AA.  AD,  Ae,  and 
At.)  If  significant  discrepancy  exists,  the  patterns  of  this  discrepancy  with  respect  to  various 
milestones  in  the  test  run  log  may  be  examined.  One  may  ask  what  the  effect  of  recali¬ 
bration,  breakdown  and  reconfiguration  of  equipment,  increasing  separation  of  source 
and  measurement  system,  and  so  forth  is  on  this  pattern  of  (H^  -  Hy)  values. 

While  the  results  of  such  an  examination  are  not  mathematically  conclusive,  they  may 
point  to  areas  which  should  be  investigated  by  the  measurement  team.  For  instance,  suppose 
all  the  values  of  (H^  -  Hy)  were  positive  following  calibration  R  j,  and  all  negative  following 
calibration  Ri.  In  spite  of  the  errors  AT,  AA,  AD.  Ae,  At,  which  are  not  known,  this  would 
be  a  strong  indication  that  the  calibration  procedure  was  biasing  the  measured  result. 
This  type  of  analysis  tor  the  blackbody  data  is  presented  in  the  section  of  this  report  entitled 
Comparison  of  Measured  and  Calculated  Values. 


•This  bias  is  noi  to  he  confused  with  the  bias  which  may  exist  in  the  measurement  of  the  defining  parameters 
which  is  a  separate  problem  entiiely. 
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Data  Discrepancies  Among 
Different  Measurement  Systems 


Wlit'n  an  experiment  is  performed  using  one  measurement  system,  and  then  repeated 
using  a  different  measurement  system,  even  greater  discrepancies  may  result  among  the 
data  items  than  when  the  same  measurement  system  is  used  repeatedly.  In  attempting  to 
reproduce  the  experiment  there  are  now  two  variations  to  be  accounted  for  the  variation 
of  calculated  irradiance  about  true  irradiance  at  both  facilities  and  the  unexpected  differ¬ 
ences  in  true  irradiance  at  the  two  facilities.  If  ll^j  and  Hj|  denote  calculated  and  true 
irradiance  at  the  first  facility,  and  and  U  p  denote  calculated  and  true  irradiance 
at  the  second  facility,  the  reproducibility  of  the  experiment  can  be  defined  in  temis  of  the 
standard  deviation  of  the  distribution  of  the  second  configuration  about  the  first  as 


a  =  lo' 


(II 


Cl 


-  ht1> 


+  o' 


(II 


c: 


In  addition,  the  bias  effects  and  the  sample-to-sample  variabilities  from  both  facilities  will 
affect  the  variability  of  data  items  from  the  experiments. 

The  application  of  the  concepts  of  data  variability  developed  in  this  chapter  to  the 
blackbody  measurement  data  collected  at  China  Lake  and  White  Sands  is  described  in  the 
next  section  of  this  report.  The  concepts  developed  in  this  chapter  are  summarized  in  Table  2. 

Convention  for  Describing 
Measurement  Discrepancies 


The  terminology  used  in  this  report  will  follow  the  conventions  that 

1.  Uncertainty  refers  to  the  interval  over  which  data  from  one  facility  can  range  for 
the  same  assumed  physical  conditions. 

2.  Reproducibility  refers  to  the  interval  over  which  data  from  one  facility  can  range 
due  to  human  inability  to  accurately  measure  all  the  defining  parameters. 

3.  Variability  describes  the  effects  introduced  by  the  probabilistic  distribution  of 
measurement  values  fora  given  measurement  system. 
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Defining  equation  from  Analytical  properties  of  Requires  knowledge  of  errors  in 

physical  laws  (Planck’s  defining  equation  physical  parameters  which  define 
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ANALYSIS  OF  EXPERIMENTAL  DATA 


DATA  ACQUISITION 
Acquisition  Methodology 

The  goal  of  the  data  collection  effort  was  to  acquire  data  representative  of  the  output 
of  the  measurement  facilities  including  factors  unique  to  the  measurement  equipment, 
the  dedicated  computer  hardware,  the  particular  data  reduction  computer  programs  used, 
and  the  personal  equations  of  the  members  of  the  measurement  teams.  For  this  reason, 
editing  of  the  reduced  data  was  permitted,  but  only  for  the  purpose  of  eliminating  output 
which  clearly  did  not  correspond  to  a  proper  interferometer  scan  in  the  opinion  of  the 
editors.  This  same  editing  process  is  routinely  applied  by  the  facilities  to  data  reported 
from  their  regular  measurement  programs,  and  is  therefore  appropriate  for  this  study. 

For  each  temperature/aperturc/distance  combination,  approximately  100  interfero¬ 
meter  scans  were  recorded  in  analog  format  on  magnetic  tape.  The  distance  from  the  black- 
body  radiation  source  to  the  measurement  apparatus  was  measured  with  surveyor's  tape. 
The  interferograins  were  translated  to  spectral  irradiance  by  the  dedicated  computer 
equipment.  The  measured  distance  was  used  to  compute  spectral  radiant  intensity  on  the 
computer.  The  computer  output  consisted  of  both  digital  and  graphical  representations 
of  spectral  radiant  intensity  (see  Figures  1.  2.  and  3). 

Except  for  cases  where  editing  removed  an  output  graph  from  the  sequence  of  scans. 
30  sequentially  numbered  scans  were  used,  l  or  the  two  facilities,  these  were  designated  as: 

China  Lake  Seans  3-32 

While  Sands  Scans  10-39 

This  arbitrary  selection  was  chosen  to  preclude  the  judicious  choice  of  the  "best"  30  out 
of  100  scans  by  a  facility  to  improve  the  apparent  variability  of  reported  data.  Substitutions 
for  edited  scans  were  required  to  be  made  in  sequence  following  the  last  scan  requested 
until  a  sample  of  30  scans  was  reported.*  The  integrated  value  of  spectral  radiant  intensity 
from  3.5pm  to  4.0pm  was  calculated  during  the  computer  processing  and  reported  for  each 
scan. 


China  Lake  Equipment  Configuration 

The  radiation  sou  rev  used  in  the  China  Lake  portion  of  the  experiment  was  a  black- 
body  with  a  I /2-inch  diameter  cavity.  This  source  was  positioned  at  the  focal  point  of  the 
optics  of  a  parabolic  reflector  of  25-inch  focal  length  with  a  5-inch  diameter  mirror.  Four 
stops  were  placed  across  the  collimated  beam  of  radiation  to  provide  a  range  of  aperture 
sizes.  A  range  of  three  nominal  temperatures  was  covered.  The  source  was  observed  with 
a  (Iener.il  Dynamics  Model  PI  S-I0I  Michelson  interferometer  from  distances  of  100. 
200,  and  500  feet.  The  measurement  system  field-of-vicw  was  4.5  degrees. 

♦Some  of  the  dalii  vis  wore  delivered  with  fewer  Ilian  to  members.  These  da  la  sets  were  included  in  the  studs  as 
reerived.  due  10  the  crisis  associated  with  acquiring  (he  additional  ic<|  lies  ted  scans 
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White  Sands  Equipment  Configuration 

The  data  collection  effort  performed  by  OMI’W  was  conducted  at  Mar  Site  on  the 
Anns  White  Sands  Missile  Range.  New  Mexico.  The  IR  radiation  source  used  was  a  black- 
body  with  a  3-inch  diameter  aperture.  Apertures  of  varying  size  were  placed  in  front  of  the 
blackbody  aperture  to  provide  the  range  of  values  for  that  parameter.  The  blackbody  tem¬ 
perature  was  'hcmiostatically  controlled.  The  reported  temperature  was  recorded  from  a 
digital  readout  to  one  decimal  place  associated  with  a  temperature  probe  inserted  in  the 
cavity  of  the  blackbody  distinct  from  the  temperature  probe  associated  with  the  thermo¬ 
static  control  The  source  was  observed  with  a  General  Dynamics  Model  PFS-201  Michelson 
interferometer  located  at  a  distance  of  500  feet  from  the  blackbody.  The  collecting  optics 
of  the  interferometer  had  a  field-of-view  of  0.6  degrees. 


ANALYSIS  OF  THE  MEASURED  VALUES 
Statistical  Analysis 

Some  basic  characteristics  of  the  measured  data  samples  have  been  developed  in  ac¬ 
cordance  with  the  formulas  presented  in  the  Appendix  to  this  report.  These  values  are  listed 
for  the  China  Lake  data  in  Table  3  and  for  the  White  Sands  data  in  Table  4.  For  each  data 
sample,  a  calculated  value  of  irradiance  is  given,  derived  from  application  of  Planck's  Law 
to  the  measured  values  of  temperature,  aperture,  and  distance  for  the  data  cell.  The  mean 
of  each  data  sample  is  given  as  the  measured  value  of  irradiance.  The  standard  deviation  is 
descriptive  of  the  spread  of  the  sample  about  the  mean. 

Tests  for  Normalcy 

One  way  of  determining  whether  a  normal  distribution  represents  a  good  fit  forgiven 
data  is  to  use  probability  (Gaussian)  graph  paper.  The  percentage  cumulative  frequency 
of  the  data  sample  is  plotted  as  a  function  of  the  measured  parameter.  The  degree  to  which 
all  the  plotted  points  lie  on  a  straight  line  determines  the  closeness  of  fit  of  the  given  distri¬ 
bution  to  a  normal  distribution.  This  method  has  been  used  to  determine  the  normalcy  of 
the  irradiance  data  samples  by  inspection.  A  sample  plot  is  shown  in  Figure  10. 

Analysis  of  Skewness  and  Kurtosis 


Skewness  describes  the  degree  to  which  a  distribution  departs  from  a  symmetric  con¬ 
figuration.  The  moment  coefficient  of  skewness,  defined  as  the  third  moment  about  the 
mean  divided  by  the  cube  of  the  standard  deviation  (see  Appendix),  has  a  value  of  zero  for 
perfectly  symmetrical  curves  such  as  the  normal  curve.  If  the  computed  skewness  of  a  distri¬ 
bution  is  positive,  the  distribution  has  a  longer  “tail”  to  the  right  of  the  maximum  of  the 
curve.  If  the  skewness  is  negative,  there  is  a  longer  “tail"  to  the  left  of  the  maximum. 
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Table  3.  Characteristics  of  the  China  Lake  Data  Samples. 


i 


Distance, 

in 

Tempeiature. 

°K 

Aperture. 

cm- 

Calculated 
value  of 
irradiance 

Mean  measured 
value  of 
irradiance 

Standard 
deviation 
of  sample 

15:. 40 

804.4 

27 

F.-08 

2.24 

F-08 

7.06 

F-10 

126.7 

3.74 

1-08 

3.60 

F-08 

8.43 

1-M0 

5.067 

3.65 

1-09 

2.65 

F-09 

3.25 

F-10 

B  .  B 

29.2 

2.10 

F-08 

1.71 

F-08 

6.55 

F-10 

77.07 

5.55 

F-08 

4.73 

F-08 

9.90 

F-10 

5.067 

6.01 

E-09 

5.39 

F-09 

3.58 

F-10 

1 142.7 

■> 

3.64 

F-08 

3.35 

F-08 

7.85 

F-10 

1 143.2 

9.63 

F-08 

8.98 

F-08 

2.37 

F-09 

1148.2 

126.7 

1.61 

F.-07 

1.46 

F-07 

2.26 

F-09 

1 126.2 

6.01 

F-09 

5.67 

F-09 

4.29 

F-10 

1  142.7 

29.2 

3.64 

H-08 

3.56 

F-08 

7.21 

F-10 

1 143.2 

9.63 

F-08 

9.52 

F-08 

2.51 

F-09 

1  148.2 

126.7 

1.61 

F-07 

1.55 

F-07 

2.53 

F-09 

60.96 

1 

5.067 

9.44 

F-09 

9.45 

F-09 

7.19 

F-10 

29.2 

5.44 

F-08 

5.29 

E-08 

8.78 

F-10 

80(i.2 

72.45 

1.35 

F-07 

1.26 

F-07 

1.52 

F-09 

806.2 

101.3 

1.89 

i.  27 

1.76 

F-07 

1.98 

F-09 

993.2 

5.067 

2.35 

F-08 

2.34 

F-08 

6.09 

F-10 

988.2 

29.2 

1 .33 

F-07 

1.30 

F-07 

1 .36 

F-09 

994.2 

101.3 

4.71 

F-07 

4.47 

F-07 

3.12 

F-09 

1 140.2 

5.067 

3.92 

F-08 

4.11 

F-08 

1.01 

F-09 

1  145.7 

29.2 

2.30 

1-07 

2.28 

F-07 

2.33 

F-09 

1 141.5 

101.3 

7.87 

F-07 

7.77 

F-07 

9.9| 

F-09 

30.48 

5.067 

3.75 

F-08 

3.76 

F-08 

1.43 

F-09 

29.2 

2.16 

F-07 

2.16 

F-07 

1.67 

F-09 

77.07 

5,71 

F-07 

5.84 

F-07 

2.89 

F-09 

126.7 

9.38 

F-07 

8.34 

F-07 

2.94 

F-09 

994.2 

5.067 

9.42 

F-08 

9.65 

F-08 

1.22 

F-09 

993.2 

29.2 

5.41 

F-07 

5.39 

F-07 

3.49 

F-09 

990.2 

1.41 

F-06 

1.44 

F-06 

3.97 

F-09 

990.2 

2.32 

F-06 

2.09 

1-06 

6.43 

F-09 

30.48 

1  143.2 

5.067 

1.58 

F-07 

1.69 

F-07 

1.30 

F-09 

1 143.2 

29.2 

9.12 

F-07 

9.1  1 

F-07 

3.14 

F-09 

1 154.7 

2.49 

K-06 

2.57 

F-06 

7.87 

F-09 

1160.7 

4.17 

F-06 

3.79 

F-06 

8.91 

F-09 
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Table  4.  Characteristics  of  the  White  Sands  Data  Samples. 


Distance, 

m 


Temperature. 

°K 


Aperture, 

•> 

cm- 


Calculated 
value  of 
inadiance 


Mean  measured 
value  of 
irradiancc 


Standard 
deviation 
of  sample 
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Kurtosis  ilescrihes  the  ‘'peakedness''  of  a  distribution.  The  moment  coefficient  of 
kitrtosis.  defined  as  the  fourth  moment  about  the  mean  divided  by  the  fourth  power  of  the 
standard  deviation  (see  Appendix),  has  a  value  of  three  for  the  normal  curve.  The  normal 
curve  is  described  as  moderately  peaked,  or  mesokurtic.  For  values  greater  than  three,  a 
distribution  becomes  more  peaked,  or  leptokurtic.  For  values  less  than  three,  a  distribution 
is  (latter,  or  platvkurtic.  The  concepts  of  skewness  and  kurtosis  are  illustrated  in  Figure  1  1. 


Pos i t i vc  Skewness 

A 

Negative  Skewness 

Zero  Skewness 

Leptokurtic 

Platykurtic 

Mesokurtic 

Figure  1 1 .  Skewness  and  Kurtosis. 


For  a  data  sample  of  size  N.  unbiased  estimates  of  the  moment  coefficient  of  skewness, 
g],  and  the  excess  of  the  kurtosis  over  that  of  the  normal  curve.  g->.  may  be  computed.* 

It  can  be  shown  that  if  all  possible  samples  of  size  N  were  drawn,  all  the  estimates 
of  gj  would  be  distributed  with  a  standard  deviation  of 

S  =  (6/N>- 
1 

and  all  the  estimates  of  gi  would  be  distributed  with  a  standard  deviation  of 
S^CO/N)’- 

These  facts  have  been  used  to  test  the  irradiancc  data  samples  analytically  for  normalcy. 
Unbiased  estimates  of  skewness  and  excess  have  been  developed  for  all  the  data  samples. 
These  estimates  have  been  compared  with  the  standard  deviations  for  their  distributions. 
Ml  data  samples  having  estimates  of  skewness  and  excess  within  three  standard  deviations 
of  the  values  for  the  normal  curve  will  be  described  as  well-fitted  by  a  normal  distribution. 

The  estimates  of  skewness  and  excess  for  the  data  samples  are  given  in  Table  5  for 
China  I  ake  and  Table  6  for  White  Sands.  Also  given  arc  the  numbers  of  standard  deviations 
by  which  these  estimates  exceed  those  for  the  normal  distribution. 


*Soc  Mathematics  "I  Slalishcs.  Pan  Two,  hy  J.  I  .  Kenney  ami  I .  S.  Keeping.  Van  Nnsiraml.  195  I,  pp.  109-1 10 
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Table  5.  Unbiased  estimates  of  the  Skewness  ami  l  \eess  I'm  China  l  ake 

Data  Samples. 


Distance. 

m 

Temperature, 

°K 

Aperture, 

cm- 

Estimate 

of 

skewness 

Skewness 
in  standard 
deviations 

estimate 

of 

excess 

Ixcess 
in  standard 
deviations 

152.40 

804.4 

77.07 

-1.287 

-0.214 

-0.245 

804.7 

126.7 

0. 1 74 

-0.002 

i)86.2 

5.067 

0.867 

1 .458 

0.145 

485. 7 

24.2 

0.640 

-0.557 

-0.577 

486.2 

77.07 

-0.556 

-0.747 

-0.544 

1126.2 

5.067 

0.074 

0.165 

-0.446 

1 142.7 

24.2 

-0.200 

-0.447 

1.002 

1.120 

1 145.2 

77.07 

-1.205 

-2.640 

2.650 

2.441 

1 148.2 

126.7 

-0.247 

-0.555 

0.541 

0.457 

1 126.2 

5.067 

-0.182 

-0.407 

-0.445 

-0.552 

1 142.7 

24.2 

0.408 

0.412 

mmm 

1 145.2 

77.07 

■  1.078 

-2.41 1 

■fH 

2.666 

1 148.2 

126.7 

-0.051 

-0.064 

si 

-0.142 

60.% 

806.2 

5.067 

0.560 

0.805 

-0.515 

-0.574 

806.2 

24.2 

-0.074 

-0.166 

-1.051 

-1.175 

806.2 

72.45 

-0.515 

-0.704 

-0.488 

-0.545 

806.2 

101.5 

0.476 

1.064 

0.445 

0.554 

445.2 

5.067 

0.070 

0.157 

-0.844 

-1.005 

488.2 

24.2 

0.260 

0.581 

0.156 

0.152 

444.2 

101.5 

0.151 

0.554 

-0.604 

-0.675 

1  140.2 

5.067 

-0.21 1 

-0.471 

-0.460 

•0.5 1 5 

1  145.7 

24.2 

0.182 

0.408 

-1.122 

-1.255 

1  141.5 

101.5 

1 .046 

2.525 

1.727 

1.852 

50.48 

805.2 

5.067 

-0.251 

-0.562 

-0.401 

-1.007 

805.2 

24.2 

-0.255 

-0.525 

-0.726 

-0.812 

805.2 

77.07 

-0. 1 1 5 

-0.258 

0.515 

0.550 

805.2 

126.7 

0.205 

0.454 

-0.587 

-0.657 

444.2 

5.067 

0.5  5  7 

0.744 

0.45 1 

0.482 

445.2 

24.2 

-0.545 

-0.766 

0.485 

0.542 

440.2 

77.07 

-0.444 

-0.444 

-0.616 

•0.684 

440.2 

126.7 

-0.561 

•0.766 

-0.241 

-0.255 

1 145.2 

5.067 

-0.268 

-0.544 

-0.65  1 

-0.728 

1145.2 

24.2 

0.186 

0.417 

-0.454 

-0.515 

1154.7 

77.07 

-0.711 

-1.584 

0.440 

0.442 

1  160.7 

126.7 

0.148 

0.445 

■1.004 

-1.122 
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['able  6.  Unbiased  l  stimates  of  the  Skewness  and  Fxecss  for  White  Sands 

Data  Samples. 


Distance. 

m 

Temperature. 

°K 

Aperture, 

*> 

cm- 

fstimaie 

of 

skewness 

Skewness 
in  standard 
deviations 

fstimate 

of 

excess 

f  xccss 
in  standard 

deviations 

1  55.0 

950.26 

0.713 

0.221 

0.493 

-0.685 

-0.766 

950.26 

5.067 

0.235 

0.526 

-0.495 

-0.553 

950.26 

20.27 

0.480 

1.055 

-0.264 

•0.290 

950.26 

45.6 

0.677 

1.514 

-0.022 

-0.025 

11  55.86 

0.713 

0.715 

1.599 

0.580 

0.649 

11 55.86 

5.067 

-0.299 

-0.669 

0.462 

0.517 

1155.86 

20.27 

■0.63  1 

-1.410 

0.995 

1.112 

1 155.86 

45.6 

0.401 

0.882 

1.049 

1 . 1 54 

1422.76 

20.27 

-0.002 

-0.004 

-0.331 

-0.370 

I42.V76 

45.6 

0.976 

2.181 

1.596 

1.785 

Kolmogorov-Smirnov  Test  for  Normalcy 

In  the  two  preceding  sections,  methods  of  data  inspection  were  described  which  enable 
an  analyst  to  determine  whether  a  normal  distribution  is  a  reasonable  description  of  meas¬ 
urement  data.  To  give  credibility  to  the  assumption,  however,  an  analytical  test  for  good¬ 
ness  of  fit  between  the  measurement  data  and  a  particular  normal  curve  should  be  applied. 
The  Kolmogorov-Smirnov  test  is  just  such  an  analytical  procedure  for  testing  goodness  of 
fit  between  empirical  data  and  a  specified  theoretical  distribution. 

In  order  to  apply  this  method  as  a  test  for  normalcy,  a  theoretical  normal  curve  must 
be  specified.  This  is  done  by  using  the  sample  mean  and  sample  standard  deviation  as  point 
estimates  of  the  population  characteristics.  The  numerical  sample  values  for  mean  and 
standard  deviation  are  used  in  place  of  p  and  o.  respectively,  in  the  equation  for  the  normal 
density  function 

V  = — I — exp  li  x  *  £  * 
ox/Zrr  la2 

The  normal  ogive  curve  for  the  specified  equation  is  plotted.  Dashed  curves  are  plotted 
above  and  below  the  ogive  corresponding  to  some  preselected  level  of  significance  to  create 
a  "decision  band”  within  which  the  specified  equation  is  accepted.  (A  significance  level 
ol  a  implies  that  100  of/t  of  the  time  this  test  will  reject  the  curve  fit  when  it  is  an  appropri¬ 
ate  one.)  The  measurement  data  are  plotted  on  the  same  graph,  and  if  all  the  data  points 
lie  within  the  decision  band,  the  normal  curve  fit  is  accepted.  Figure  1 2  is  an  illustration 
of  the  Kolmogorov-Smirno’  test  applied  to  one  of  the  C  hina  Lake  data  samples.  This  sample 
had  the  largest  moment  coefficient  of  skewness  and  excess  of  kurtosis.  All  the  data  points 
lie  within  the  decision  band  and  the  assumption  is  accepted. 
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Figure  12.  Kolmogorov-Smimov  Test  of  Normal  Ogive  Curve  for  China  Lake  Data  Set 
With  80%  Confidence  Level 


A  Mathematical  Model 
for  Data  Variability 
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The  data  matrix  upon  which  this  study  is  based  consists  of  45  data  cells,  I'ach  cell 
is  determined  by  three  unique  values  true  temperature,  true  aperture,  and  true  dis¬ 
tance  which  correspond  to  a  unique  true  value  of  iradiance  Hy.  Within  each  cell  is  a 
sample  of  about  30  measured  values  of  irradiance  drawn  from  an  infinite  population 
of  possible  measurement  values  for  the  cell.  The  sample  within  each  data  cell  has  been 
characterized  by  the  following  statistics: 

the  sample  mean 

Sii,  the  sample  standard  deviation 
V i.i.  The  sample  coefficient  of  variation* 

These  values  arc  point  estimates  of  the  characteristics  of  the  infinite  population  of  possible 
measurement  values  for  the  cell  from  which  the  sample  was  drawn.  The  characteristics  of 
the  infinite  population  are: 

jU||,  the  population  mean 

0||.  The  population  standard  deviation 

the  population  coefficient  of  variation* 

Values  are  given  for  the  coefficients  of  variation  for  the  China  Lake  data  samples  in  Table  7 
and  for  the  White  Sands  data  samples  in  Table  8.  Standard  deviation  values  are  listed  for 
convenience.  The  values  listed  within  the  tables  for  the  coefficients  of  variation  are  to  be 
used  with  the  factor  10'*-  as  indicated  by  the  column  heading.  Therefore  they  may  be  read 
directly  as  percentage  variation. 

The  percentage  variation  differs  from  cell  to  cell  and.  in  general,  decreases  with  in¬ 
creasing  mean  measured  irradiance.  A  mathematical  model  relating  measurement  variability 
to  the  measured  parameter  will  now  be  proposed  as  an  explanation  for  the  behavior  of  the 
data. 


The  noise  in  an  IR  measurement  system  can  be  described  in  three  basic  categories: 

1.  Noise  due  to  variations  in  the  background  against  which  the  measured  object 
is  viewed. 

2.  Noise  due  to  radiation  within  the  system  optics. 

3.  Thermal  detector  noise. 

4.  Noise  due  to  random  effects  associated  with  the  circuitry  which  transforms 
the  detector  response  to  a  system  output. 


The  first  type  of  noise  can  be  reduced  by  various  discrimination  techniques,  and  the  second 
and  third  by  cooling  the  system.  The  fourth  type  is  a  characteristic  of  the  measurement 
system  and  will  respond  only  to  system  modification. 

•Torffine.il  of  variation  i%  one  measure  of  the  relative  dis|»crsion  of  a  sample  or  a  population.  It  is  defined  as  the 
*f  amfurcJ  deviation  drvnJi'tJ  h>  the  mean. 
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Table  7.  Variability  of  China  Lake  Data. 


Distance, 

m 

Temperature, 

°K 

Aperture. 

■> 

cm- 

Mean  measured 
irradiance. 
watts/eni-  x  10"'® 

Standard  deviation 
in  watts/cm- x  10'*® 

Coefficient 
of  variation, 
pure  x  10'- 

152.40 

77.070 

224.36 

7.058 

3.15 

1  26.700 

359.82 

8.430 

2.34 

986.20 

5.067 

26.50 

3.249 

1 2.26 

985.70 

29.200 

171.14 

6.550 

3.83 

986.20 

77.070 

472.52 

9.895 

2.09 

1 126.20 

5.067 

53.89 

3.584 

6.65 

1 142.70 

334.98 

7.852 

2.34 

1 143.20 

■ 

898.02 

23.715 

2.64 

1 148.20 

1  26.700 

1462.2 

22.582 

1.54 

1 1 26.20 

5.067 

56.68 

4.294 

7.58 

1 142.70 

29.200 

356.46 

2.02 

1 143.20 

77.070 

95 1 .69 

2.63 

1 148.20 

1 26.700 

1549.3 

iiillS 

1.63 

60.96 

806.20 

5.067 

94.47 

7.187 

7.61 

806.20 

29.200 

528.70 

8.78.7 

1 .66 

806.20 

72.450 

1263.5 

15.216 

1.20 

806.20 

101.300 

1762.4 

19.829 

1.12 

5.067 

233.70 

6.093 

2.61 

29.200 

1303.4 

13.581 

1.04 

' 

W4.20 

101.300 

4468.5 

31.233 

.70 

60.% 

1 140.20 

5.067 

41  1.06 

10.130 

2.46 

1145.70 

29.200 

2283.3 

23.335 

1.02 

1 141.50 

101.300 

7770.4 

99.148 

1.28 

30.48 

805.20 

5.067 

375.55 

14.295 

3.81 

805.20 

29.200 

2157.9 

16.684 

.77 

805.20 

77.070 

5837.1 

28.900 

.50 

805.20 

126.700 

8338.9 

29.352 

.35 

994.20 

5.067 

965.23 

12.191 

1.26 

993.20 

29.200 

5387.1 

34.914 

.65 

990.20 

77.070 

14352. 

39.657 

.28 

990.20 

126.700 

20855. 

64.258 

.31 

1143.20 

5.067 

1687  4 

12.995 

.77 

1 143.20 

29.200 

9105.2 

3 1 .395 

.34 

rjK-Vy- 
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Table  8.  Variability  of  White  Sands  Data. 


Distance, 

in 

Temperature. 

°K 

Aperture, 

A 

cm- 

Mean  measured 
iriadiatice. 
waits/cin-  \  10' 'f* 

Standard  deviation 
in  waitVctn- x  10' 10 

Coefficient 
of  variation, 
pure  x  10'- 

1 55.0 

‘>50.2(1 

0.713 

4.45 

.177 

3,‘)‘> 

‘>50.26 

5.067 

3 1 .  7*> 

.381 

1.20 

‘>50.26 

20.270 

124.34 

W) 

.73 

‘>50.26 

45.600 

274. ‘>7 

,8‘>2 

.32 

1 155.86 

0.713 

‘>.18 

.172 

1.87 

1155,86 

5.067 

64.28 

.720 

1.12 

1 155.86 

20.270 

257.12 

2.725 

1.06 

1155.86 

45.600 

571.76 

3.426 

.60 

1423.76 

20.270 

486.78 

3.514 

.72 

1423.76 

45.600 

1148.8 

8.100 

.70 

The  postulate  is  stated  that  for  any  individual  measurement 


HM  -  HT  +  tp  +  tN 


where 


is  the  measured  value  of  irradiauce 
Hj  is  the  true  value  of  ir'adiance  (an  unknown  constant) 
cp  and  e|Nj  arc  statistically  independent,  random  variables  such  that 
CP=  N|<0/'HtI*  and 
eN  =  N|f("B)."Nl 

where  Up  is  the  irradiance  due  to  background  effects. 

The  error  term  e|>  is  descriptive  of  the  measurement  system  behavior  with  respect  to 
the  magnitude  of  the  measured  parameter,  and  in  fact  the  standard  deviation  of  the  possible 
infinite  population  of  c|>  values  is  proportional  to  the  measured  parameter  by  a  constant 
coefficient  p.  The  error  term  cjg  includes  the  noise  which  is  inherent  in  the  procedure  no 
matter  what  the  value  of  the  measured  parameter  is.  All  system  bias  is  included  in  the  mean 
value  of  the  infinite  population  of  possible  values. 

•THii  is  read  "r  p  is  normally  distributed  with  a  mean  ol  0  and  a  standard  deviation 
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The  noise  is  the  result  of  all  contributing  physical  factors  characteristic  of  the  system, 
and  is  of  the  form 

eN  =  CNI  +eN2  +  tN3  +  -  •  ■ 

where  the  e^j  are  all  statistically  independent,  random  variables,  but  this  does  not  alter 
the  generality  of  the  present  model  development.  However,  it  does  indicate  the  potential 
for  noise  reduction  by  identifying  and  reducing  contributing  system  effects. 

The  measurement  variance  oj|*  of  an  infinite  population  of  possible  measurements 
Hjy|  can  be  expressed  as  the  sum  of  the  variances  of  the  error  terms 

oH2  =  p2Hy2  +on2 

since  Ht  is  a  constant.  Furthermore,  if  we  denote  the  mean  of  the  noise  distribution, 
which  we  have  included  to  account  for  bias  due  to  background  effects  and  system  charac¬ 
teristics,  by  the  constant  B,  then 

jU|  |  =H-|-  +  B,  or  H"|-  =  pj.|  -  B 

Values  of  sample  standard  deviation,  S|.j,  and  mean  measured  irradiance,  have  been 
used  as  point  estimates  of  0||  and  mjj  in  graphing  this  relationship.  The  graphs  for  the  China 
Lake  data  are  shown  in  Figures  13,  14,  and  15.  The  White  Sands  data  are  graphed  in  Figure  16. 

The  postulation  of  this  mathematical  model  for  measurement  variance  in  an  1R  meas¬ 
uring  device  is  not  unrealistic.  The  mathematical  expression  for  error  proportional  to  the 
measured  parameter  has  been  applied  to  other  measurement  processes,  one  such  being 
electrical  measurements,  in  which  the  noise  term  is  the  familiar  “Johnson  noise",  where 
noise  power  is  proportional  to  resistance  and  absolute  temperature  over  a  limited  range  of 
observations.* 


Background  Effects  and  System  Noise 


The  acceptance  of  the  mathematical  model  postulated  in  the  preceding  section  as  an 
appropriate  description  of  the  measurement  variance  experienced  during  the  use  of  these 
interferometers  will  depend  on  future  observations.  The  data  matrix  used  for  the  present 
study  is  too  small  to  yield  conclusive  results.  In  particular,  a  much  more  detailed  data  base 
would  be  needed  at  a  low  value  of  irradiance  to  accurately  determine  the  limiting  value 
ofS„  as  irradiance  decreases. 

If  the  postulated  model  is  accepted  and  sample  statistics  are  used  as  estimates  of  the 
measurement  population  characteristics,  the  relationship  has  the  form 

sH2=^<nM-B)2+SN2 

*1  or  h  discussion  ol  Johnson  noise,  see  Reference  Data  lot  Radio  I  nginccr*.  Howard  W.  Sams  &  Co.,  1975.  Arti¬ 
cle  17  12  - - - 


Figure  13.  Sample  Standard  Deviation  as  a  Function  of  Mean 
Measured  Irradiance,  China  Lake  Data  Measured  at 
1 52.4  Meters. 
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Figure  14.  Sample  Standard  Deviation  as  a  Function  of  Mean  Measured  Irradiance, 
China  Lake  Data  Measured  at  60.96  Meters. 
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Figure  IS.  Sample  Standard  Deviation  as  a  Function  of  Mean  Measured  Irradiance, 
China  Lake  Data  Measured  at  30.48  Meters. 
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Figure  16.  Sample  Standard  Deviation  as  a  Function  of  Mean  Measured  Irradiance, 

White  Sands  Data  Measured  at  155  Meters. 
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When  this  is  rewritten  as 


csn/p>2 


it  has  the  form  of  a  hyperbola  with  branches  above  and  below  the  Hj^  axis  whose  center 
is  the  point  (B,0).  In  actual  practice 

HM>B,andSH>0 

so  that  only  the  right  half  of  the  upper  branch  of  the  hyperbola  could  ever  be  generated 
by  measurement  data  as  shown  in  Figure  17. 

The  value  of  Sjq  is  the  minimum  value  of  Su  which  can  be  obtained  as  is  decreased. 
The  limiting  value  of  the  slope  of  the  curve  as  Rj^  is  increased  (which  is  the  slope  of  the 
asymptote)  is  p.  The  value  of  B  may  be  estimated  from  the  intersection  of  the  hyperbolic 
asymptote  with  the  axis  of  abscissas.  These  mathematical  concepts  could  be  used  by 
measurement  facilities  to  estimate  the  standard  deviation  of  the  noise  error,  the  background 
term,  and  the  fractional  coefficient  in  the  percentage  error  term  for  their  measurement 
equipments. 

Coefficient  of  Variation 

The  equation  developed  to  relate  measurement  variance  to  mean  measured  irradiance 

V  =  p*<nM  -  b)2  +  s2 


may  be  written 


2RmB  +  B2 


s  2 

+  X 

V 


which  in  terms  of  the  coefficient  of  variation  Vjj  gives 


VH 


.  -,23.  +  JL 

"m  hm2 


s  2 

+  x 

H  NI¬ 


AS  decreases  to  B,  the  first  term  approaches  0,  and  the  value  of  V ^  |  tends  toward  the 
coefficient  of  variation  due  to  noise. 

For  >>  B  (and  consequently  >>Sj^),  the  value  of  approaches  p,  which 
is  the  postulated  fractional  error  in  measurement  of  the  true  irradiance.  Plots  of  coefficient 
of  variation  as  a  function  of  mean  measured  irradiance  are  therefore  a  means  for  determin¬ 
ing  the  value  of  p  by  inspection.  These  plots  are  given  in  Figures  18,  19,  and  20  for  the 
China  Lake  data  and  in  Figure  21  for  the  White  Sands  data. 
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Figure  1 8.  Sample  Coefficient  of  Variation  as  a  Function  of  Mean  Measured 
Irradiance,  China  Lake  Data  Measured  at  1 52.4  Meters. 
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Figure  19.  Sample  Coefficient  of  Variation  as  a  Function  of  Mean  Measured 
Irradiance,  China  Lake  Data  Measured  at  60.96  Meters. 
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Figure  20.  Sample  Coefficient  of  Variation  as  a  Function  of  Mean  Measured 
Irradiance,  China  Lake  Data  Measured  at  30.48  Meters. 
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Figure  21.  Sample  Coefficient  of  Variation  as  a  Function  of  Mean  Measured 
Irradiance,  White  Sands  Data  Measured  at  155  Meters. 
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Computing  Percentage  Variability 
and  System  Noise 

If  the  mathematical  statement 

sh’  =  p2(Rm-b)2+sn2 

is  a  reasonable  description  of  the  measurement  variability,  and  if  we  can  assume  that  both 
background  and  system  bias  are  very  small,  we  may  write 

sHJ  =  +  sn2 

A  graph  of  Spj2  as  a  function  of  Hj^2  would  then  be  a  straight  line  with  slope  p2  and  inter¬ 
cept  Sjvj2  .  This  straight  line  can  be  determined  analytically  bv  using  the  method  of  least 
squares  to  fit  the  data. 

A  linear  correlation  coefficient  can  be  computed  for  the  variables 

Y  =  Sh2  andX  =  HM2 

to  determine  the  appropriateness  of  the  assumption  that 

Y  =  f(X) 

is  a  straight  line.  This  correlation  coefficient  is  defined  as 

X(X  -  X)  ♦  (Y  -  Y) 
l[Z(X  -  X)2  1  (I(Y  -  Y)2  lig¬ 
and  has  values  close  to  +1  or  -1  if  a  pronounced  linear  relationship  does  exist.*  Values 
close  to  zero  indicate  almost  no  linear  correlation  between  the  variables. 

This  analysis  was  performed  on  the  35  data  cells  from  China  Lake  and  the  10  data 
cells  from  White  Sands.  The  White  Sands  data  yielded  striking  results  immediately,  with  a 
correlation  coefficient  of  .99.  First  efforts  with  the  China  Lake  data  showed  almost  no 
linear  correlation  when  all  the  data  cells  were  evaluated  together.  However,  when  the  China 
Lake  data  cells  were  evaluated  separately  by  distance,  they  showed  strong  linear  correlation. 

The  strongest  linear  correlation  shown  by  the  China  Lake  data  was  .97  at  a  distance 
of  60.96  meters.  The  correlation  for  30.48  meters  was  about  .96;  for  152.4  meters  the 
coefficient  was  about  .86. 

The  results  of  this  linear  correlation  analysis  are  listed  in  Table  9.  There  is  a  strong 
inference  that  p  is  a  function  of  distance,  which  increases  as  distance  increases.  This  suggests 
that  for  small  values  of  Hy,  Oj.j2  may  decrease  as  p2Hy2  but  take  some  minimum  “noise" 
value.  The  scope  of  the  present  measurement  program  does  not  allow  final  resolution  ot 
this  question. 


'Here  X  is  the  mean  ol  the  almisMs  and  Y  is  the  mean  of  the  ordinates  lor  the  data  points  fitted. 
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Table  9.  Fractional  Variability  From  Linear  Correlation 


of  SH2  and  Hm2. 

Distance, 

m 

Number  of 
data 
points 

Correlation 

coefficient 

_ 1 

Slope 

Fractional 

variability 

China  Lake 

30.48 

12 

0.95903 

0.2419  x  10'2 

60.96 

10 

0.97276 

1.55616  x  10‘4 

1.2474  x  10’ 2 

152.4 

13 

0.85806 

2.71528  x  10'4 

1.6478  x  10'2 

White  Sands 

152.4 

10 

0.99373 

0.49159  x  10'4 

0.7011  x  10'2 

Effects  of  Noise  on  Measurement 
System  Output 

The  effect  of  noise  relative  to  the  signal  strength  at  the  detector  of  the  measurement 
system  is  clearly  seen  in  Figures  22,  23,  and  24.  These  figures  show  graphs  of  apparent 
spectral  radiant  intensity  for  strong,  moderate,  and  weak  signals  selected  from  the  White 
Sands  data  set. 


COMPARISON  OF  MEASURED 
AND  CALCULATED  VALUES 

Significant  Systematic  Differences 

The  analysis  described  in  this  section  was  performed  to  determine  whether  a  significant 
systematic  difference  exists  between  the  true  but  unknown  mean  of  the  possible  measure¬ 
ment  population  and  the  irradiance  value  calculated  from  measured  values  of  temperature 
(T  +  AT),  aperture  (A  +  AA),  and  distance  (D  +  AD).  The  sample  means  H^j  were  used  as 
point  estimates  of  the  measurement  population  means  for  the  data  cells  in  investigating 
significant  difference.  The  emissivity  of  the  blackbody  source  cannot  be  greater  than  unity, 
but  may  be  noticeably  less  than  unity.  Emissivity  is  taken  to  be  unity  in  calculating  a 
value  of  irradiance,  so  that  a  measured  value  of  irradiance  resulting  from  an  emissivity 
less  than  unity  would  be  smaller.  Also,  a  fractional  atmospheric  transmission  factor  affects 
the  measured  value  of  irradiance,  but  is  not  accounted  for  in  the  calculated  value.  This 
would  also  tend  to  make  the  measured  value  less  than  the  calculated  value. 
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Figure  22.  Apparent  Spectral  Radiant  Intensity,  Calculated  Value  of  Irradiance 
is  1.11  B-07. 
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Figure  23.  Apparent  Spectral  Radiant  Intensity,  Calculated  Value  of  Irradiance 
is  6.36  E-09. 
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Figure  24.  Apparent  Spectral  Radiant  Intensity,  Calculated  Value  of  Irradiance 
is  4.27  E-10. 
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For  these  reasons,  the  values  of  -  H(>  were  expected  to  be  negative  and  to  indicate 
a  definite  bias  showing  calculated  irradiance  to  be  systematically  greater  than  mean  measured 
irradiance.  In  fact,  many  of  the  values  were  found  to  indicate  with  99'f  confidence  that 
a  definite  positive  bias  exists.  This  result  must  derive  from  one  or  more  of  the  following 
effects: 

1.  Error  in  the  temperature  measurement,  AT 

2.  Error  in  the  aperture  measurement,  AA 

3.  Error  in  the  distance  measurement,  AD 

4.  Error  in  the  calibrated  system  response,  AR. 

Since  the  potential  for  all  these  effects  is  present,  it  cannot  be  said  conclusively  which 
effects  are  involved.  However,  some  inferences  may  be  drawn  from  examination  of  the  data. 
The  method  will  now  be  explained. 

The  experimental  data  from  NWC  and  OMHW  were  examined  to  determine  whether 
a  significant  systematic  difference  between  measured  and  calculated  irradiance  values  truly 
exists.  This  determination  was  made  by  calculating  the  t-statistic  for  each  data  cell  and 
comparing  its  value  to  percentile  points  of  the  Student  t-distribution. 

The  assumption  is  made  that  the  random  variation  |HM  -  lV]randorn  is  normally 
distributed.  Within  each  data  cell  there  are  N  (usually  30)  measured  values  of  and  one 
calculated  value  H(-.  The  true,  but  unknown,  mean  of  the  infinite  population  of  possible 

measurements  Hu  is  uu  .  One  can  never  know  mii  ,  but  it  is  known  that 
M  M 

Mh  -  »c  =  f  |AT,AA,AD,T(D,X),e.AR(X)l 
M 

An  estimate  of  fit.  is  made  by  taking  the  mean  HM  of  the  sample  of  N  measure- 
ments  then  symmetric  confidence  limits  on  the  estimate  can  be  determined. 

The  next  step  is  to  decide  whether  the  difference  [H^  -  significantly  exceeds 
the  data  spread  introduced  by  the  random  component  of  error.  That  is,  does  a  significant 
component  [  HM  -  systematic  exist  ah°ve  and  beyond  the  variability  |  HM-  H(-l  random 

which  one  expects  measurement  system  fluctuations  to  cause?  Next  comes  the  crucial  step 
in  the  thought  process.  If  no  systematic  difference  exists,  and  the  difference  |Hj^  -  H^l 
is  completely  random,  then  =  cannot  be  rejected  as  the  true  but  unknown  mean 

of  the  infinite  population  of  all  possible  measurements  of  H^.  More  formally,  a  statistical 
hypothesis  is  formulated  that  can  be  tested  by  established  techniques. 
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The  null  hypothesis  and  its  alternative  are  stated  in  the  format  of  Neyman-Pearson: 


Null  hypothesis  Hq: 


HC  ~ 


M 


Alternative  hypothesis  H  j : 


hc^ph 


M 


A  statistic  is  defined  for  which  the  distribution  is  known  if  Hq  is  true.  This  statistic  is 

_  *\i  '  hc>/n  -  i 

S,, 


where  Su  is  the  sample  standard  deviation.  (The  mean  square  deviation  formula  forS,.2 

....  KI  ** 

was  used  in  this  computation.  Using  the  unbiased  estimator  does  not  change  the 

results.)  The  value  of  t  was  computed  for  each  data  cell  and  compared  to  the  density  func¬ 
tion  for  t’s  corresponding  to  samples  of  size  N  drawn  from  the  normal  distribution.  The 
null  hypothesis  was  rejected  for  all  cases  where  the  t-value  was  shown  to  be  greater  than 
the  values  for  99%  of  the  (-population. 

Values  of  the  t-statistic  for  the  China  Lake  and  White  Sands  data  are  given  in  Tables  10 
and  1  I  respectively,  l  or  data  samples  having  thirty  members,  the  null  hypothesis  was  re¬ 
jected  for  t-values  exceeding  2.76. 

Inspection  of  Systematic  Difference 

In  general,  any  significant  systematic  difference  between  measured  and  calculated 
irradiance  values  will  be  the  result  of  some  combination  of  factors  such  as 

e,  the  blackbody  emissivity 

r,  the  atmospheric  transmission 

which  will  tend  to  make  ll^j  less  than  H^->,  and  the  errors  in  the  measurements  of  temper¬ 
ature,  aperture,  distance,  and  system  response,  which  have  the  potential  for  affecting  the 
value  [Hm  -  H^l  in  either  a  positive  or  a  negative  sense.  Usually  it  will  not  be  possible  to 
determine  which  factors  prevail  in  a  particular  instance. 

Inspection  of  the  data  in  Table  1 2,  which  contains  f  -  H^J  values  for  the  30.48  meter 
distance  data  from  China  Lake,  suggests.  that  inspection  of  systematic  bias  patterns  mav 
be  helpful  in  some  cases.  Although  all  IHM  -  Hrl  values  might  be  expected  to  be  negative 
due  to  transmission  and  emissivity  effects,  the  data  show  a  distinct  correlation  of  bias  to 
aperture.  The  probability  (P)  of  this  suggestive  pattern  of  signs  (positive  and  negative  bias) 
resulting  from  fortuitous  circumstance  is: 
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Therefore  one  might  reasonably  postulate  that  the  first  aperture  was  cut  too  large,  the 
second  too  small,  the  third  too  large,  and  the  fourth  too  small,  and  that  these  results  pre¬ 
vailed  throughout  the  measurement  process. 


Table  10.  Values  of  the  t  Statistic  for  the  China  Lake  Data. 


Distance, 

in 

Nominal 

temperature. 

°K 

Aperture,  cm- 

5.067 

20.200 

77.070 

77.450 

101.500 

126.700 

50.48 

805 

0. 1 24 

-1.445 

24.651 

-194.72 

<>l) 96 

805 

0.07‘> 

-0.426 

• 

-50.872 

54.296 

• 

152.40 

805 

-2.200 

-9.528 

50.48 

‘WO 

10.520 

-5.555 

54.1 10 

-198.85 

60.06 

-0.852 

-8.851 

-4''  2’5 

• 

152.40 

wo 

-16.858 

-52.451 

-45.765 

50.48 

1150 

455)72 

-5.057 

54.502 

■  256.54 

60.06 

1150 

10.228 

-5.469 

-5.652 

• 

152.40 

1150 

-‘).44<> 

-20.487 

-15.058 

-55.275 

152.40 

1 150 

-4.556 

-5.005 

•2.501 

- 

•  12.620 

*Data  taken  at  aperture  in  adjacent  column. 


Table  I  !.  Values  of  the  t  Statistic  for  the  White  Santis  Data 
at  a  Distance  of  1 55  Meters. 


Nominal 

temperature. 

"K 

Measured  aperture  values  (A  +  AA).  cm- 

0.71.4 

5.007 

20.270 

45.600 

050.26 

5.408 

20.002 

17.410* 

10.745 

11 55.86 

7.045 

5. 1.4.4 

5.. 400 

-.085* 

1423.70 

-8.008 

27.870 

*Oata  let  hat  only  29  members. 


Inspection  of  systematic  bias  could  be  routinely  practiced  by  measurement  facilities 
as  a  check  on  mensural  values  of  temperature,  aperture,  and  distance.  Changes  in  bias  with 
system  recalibration  might  serve  as  a  check  on  the  response  error.  Patterns  such  as  the  one 
shown  in  Table  12  are  not  conclusive,  but  will  provide  insights  for  further  investigations 

Note  that  this  inspection  technique  is  in  reality  just  a  version  of  the  graphical  method 
explained  under  Inspection  of  Measurement  Ihiu  hr  (ira/ihiny  where  for  positive  values 
of  ihm-«cI  the  data  points  would  lie  above  the  line 


-?iv  wt 
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Table  1 2.  Values  of  |  -  H^.]  for  the  China  Lake  Data  at  a 

Distance  of  30.48  Meters. 


Nominal 


Measured  aperture  values  (A  +  AA),  cm- 


l  till  LUC, 

°K 

5.067 

29.200 

77.070 

126.700 

805 

3.3 

F-l  1 

-4.4 

E-10 

1.30 

E-08 

-1.04  H-07 

990 

2.30 

E-09 

-2.14 

E-09 

2.47 

H-08 

-2.33  F-07 

1150 

1.04 

H-08 

-1.75 

E-09 

7.85 

H-08 

-3.85  H-07 

ANALYSIS  OF  EXPERIMENTAL  THEORY 

REPRODUCIBILITY  OF  THE  EXPERIMENT 
The  Blackbody  Experiment 


In  the  two  preceding  chapters,  the  problems  of  sample-to-sample  variability  and  bias 
were  examined  with  respect  to  data  describing  blackbody  emissions.  In  this  section  the 
uncertainty  in  reproducing  the  physical  conditions  or  defining  parameters,  which  uniquely 
define  the  experiment,  is  expressed.  This  is  an  application  of  the  concepts  developed  in  the 
section  entitled  Quantifying  Reproducibility. 

The  reader  should  be  aware  that  the  mathematical  derivations  in  this  chapter  involve 
many  assumptions  whose  verifications  are  beyond  the  scope  of  this  effort.  The  analysis  is 
presented  to  show  how  the  uncertainty  associated  with  reproducing  an  experiment  can  be 
determined  from  the  defining  equation  for  the  experiment,  but  does  not  pretend  to  account 
for  all  possible  contributing  effects.  In  any  event,  the  merit  of  this  discussion  is  the  demon¬ 
stration  of  the  methodology,  since  the  actual  application  would  be  completely  different  for 
any  operating  aircraft,  which  is  the  problem  of  interest  in  the  overall  study  of  which  this 
report  forms  a  part. 


The  Defining  Equation 

Pie  defining  equation  for  reproducibility  of  the  blackbody  experiment  has  been  given 
for  the  true  value  of  irradiance  at  the  collecting  optics  of  the  measurement  system  as 


.  /*4.0  gm 

n  =  Al  I  \ 

'  5„m 


W(X,T)  t(A,D)  dA 


Acceptance  o!  this  equation  implies  that  all  problems  with  equipment  alignment  are  in¬ 
herent  in  apparent  aperture  and  that  any  variation  in  beam  strength  across  the  collimated 
beam  (for  China  Lake  data)  is  inherent  in  apparent  cmissivity. 
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Calculated  Value  of  Irradiance 


The  calculated  values  of  irradiance  for  this  experiment  were  derived  from  values  of 
unity  for  emissivity  and  atmospheric  transmission.  The  error  terms  for  these  parameters 
are  therefore  defined  as 

Ac  =  1  -  c 
At  =  1  -  r 


for  convenience  in  computation.  The  calculated  value  of  irradiance  becomes 


(A  +  AAHc  +  Ac) 
tt(D  +  AD)2 


J 


4.0  pm 

W(X,T  +  AT)*  |r(X,D  +  AD) 

,  .  +  Ar(X,D  +  AD))dX 

3.5  pm 


Two  algebraic  expressions  must  be  developed  for  use  in  obtaining  a  formula  for  calculated 
irradiance. 


Expression  1 : 


au/ 

W( X,T  +  AT)  -  W(X,T)  =  ~~r  *  AT 

a  r 


W(X.T  +  AT)  =  ~  •  AT  +  W(X/n 

o  I 


(4.0  Aim 

' 

3.5  uni 


W(X.T  +  ATHlX  =  AT 

5.5  pm 

Expression  2: 

A  +  AA  _  A  +  AA 
ir(D  +  AD)2  7T 


(4.0  pm  r 

9W  «  I 

dT  1  X  +  J 

3.5  pin  J 


I 


A  +  AA 


4.0  pm 


W(X.T)dX 


3.5  pm 


rrD2 

A  2AA1)  _AA 

»rD2  ‘  jrDJ  irl)2 


,  2AD  3AD2  4AD1  ...  ,  ...... 

I  -  +  "j^p  -  "U  ~  +  higher  order  terms  in  D  and  AD 


+  terms  of  second  order  or  more 
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If  terms  of  second  order  or  greater  ean  legitimately  be  discarded  (an  assumption  that  will 
not  be  proved  or  discussed  within  the  scope  of  this  effort)  this  becomes 

A  /  ,  2AI)  AA  \ 

Now  the  equation  for  calculated  irradiance  can  he  written 

4.0  pm 


II  =  — 
t  7T(I) 


_t_AA_  f 
+  AD)2  I 

J  3.5  , 


W(X,T  +  AT)dX 


3.5  pm 

since 

c  +  At  =  2 
r  +  Ar  =  I 

Using  the  two  expressions  just  developed,  this  can  be  rewritten 


II  = 

(  TTl)2 


2AI)  AA 
I)  A 


AT 


4.0  pm 

.  4.0  pm 

9W 

- —  dX  + 

i)T 

3.5  pm 

3.5  pm 

W(X,T)dX 


Continuing  to  discard  terms  of  second  order  or  greater,  this  becomes 

4.0  pm 


II 


<  ffl)2 


.  4.0  pm  . 

3W  1V 

AT 

—  dX  +  1 
3T  I 

j 

3.5  pm 

WtX.THIX 


3.5  pm 


2AI)  f 

D  J 


*  4.0  pm 

.  4.0  pm 

W(X.TKIX  +  ^ 

W(X,T)dX 

3.5  pm 

3.5  pm 

True  Value  of  Irradiance 

The  equation  for  true  irradiance  ean  be  rewritten 

4.0  pm 


•A”  j 


W(X.TW  1  -  Ar)dX 


3.5  pm 
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which  becomes,  if  second  order  or  greater  terms  are  discarded. 


'  4.0  jum 

"t-S?  f  ' 

_  3.5  urn 


W(X,T)dX  - 


3.  3  n  m 

4.0  jum 


4.0  M'li 

J  ' 

J  3.5  jum 


W(X.T)-ArdX 


-  Ae 


J  3.5  m 


W(X.T)dX 


m 


Difference  Between  Calculated 
and  True  Irradiance 


The  mathematical  expression  for  the  difference  between  calculated  and  true  irradiance 
(Hf  -  Hj).  based  only  on  the  assumption  that  terms  of  the  second  order  or  greater  may  be 
discarded,  is 


A 

ttD2 


I 


4.0  Min 


aw 

<rr 


dx  + 


AA 

A 


3  5  (Jiu 


\ 


4.0  Mm 

W(X.T)dX 

3  5  Min 


2AI) 

D 


.  4.0  Mui 

I  W(X.TklX  +  Ac 

J  3.5  Min 


4.0  Mm 

W(X,T)dX 

3.5  Mm 


+ 


1 


4.0  M'n 

W(X,T)ArdX 

3.5  m  Hi 


where 


W(X,T)  = 


c,/XT 


and 


aw 

ar 


c:/XT 

c,  c,  e 

XT2  .  C-./XT 

|e 
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If  the  errors  AT,  AA.  Al),  Ae.  aiul  At  which  can  occur  in  the  measurement  of  the  defining 
parameters  are  al!  statistically  independent,  random  variables,  then  the  variance  of  the 
variable  All  =  (l!{-  -  lly)  can  be  expressed  as 


n2  ...  =  o'  (k  )2  +  o'  (k  )'  +  o‘  (k_):  +  o1  (k  +  o'  (k  P 
AH  AT  f  AA  A  AD  D  Ae  e  At  t 


where  ky,  kA.  kD,  kt„  and  kT  are  the  algebraic  coefficients  of  the  errors  AT,  AA.  AD,  Ae, 
and  At,  respectively,  in  the  equation  for  (Hy  -  Hy). 


Specifically 


k  AT  - 


/-  4.0  pm 
|  c,c,  cC<MT.IX 

3.5  pm 


A 

7tD* 


kAAA  =  |  ttTV 


1 


.v.s  pm 
4.0  pm 


AT 


Xs 


dX 


«c'‘/w  n 


V'  ' 


-2A 

jrD3 


1 


3.5  pm 

4.0  pm 


■  AA 


Xs 


dX 


(e 


c./XT 


I) 


3.5  pm 


AD 


k  Ae  = 

t 


k  At  = 

T 


"4.0  pm 

A 

c,  dX 

jt|)j 

Xs  c/XT  , 

(e  '  -  1) 

3.5  pm 

r  4.0  pm 

A 

c,  At(X,I)KIX 

rrl)2 

1  ‘VSI  ,, 

J 

3.5  pm 

■Ae 


The  error  term  Ar(X.D)  in  the  last  expression  is  not  explicitly  isolated  as  the  others  are. 
It  is  still  possible  to  use  this  method  to  evaluate  variance  quantitatively,  however,  by  find¬ 
ing  the  largest  and  smallest  possible  values  lor  r  over  the  range  of  3.5  pm  to  4.0  pm  for  the 
given  distance,  and  using  these  values  to  provide  a  bracketed  value  range  for  kf  based  on 


Armin  =  1 

rmax 

Armax 

*  '  rinin 

5” 
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Numerical  Estimation  of  Reproducibility 

The  numerical  estimation  of  reproducibility  is  accomplished  by  evaluating  the  co¬ 
efficients  kj.  k^,  k|),  kf  and  kr  for  the  specified  experiment.  These  coefficient  values 
vary  as  the  values  of  the  defining  parameters  vary,  so  that  reproducibility  is  not  a  constant 
for  a  given  experimental  setup  but  changes  when  the  unique  physical  conditions  change. 

In  addition,  values  for  the  variances  o2^,  °JAI>  °JAe’  ant*  °2At  niust  'u* 

obtained.  This  is  usually  done  based  on  an  intimate  knowledge  of  the  measurement  pro¬ 
cesses  applied  to  the  defining  parameters.  The  smallest  scale  division  on  the  surveyor's 
tape  used  to  measure  distance,  the  number  of  decimal  places  in  the  digital  temperature 
readout,  and  the  characteristics  of  the  equipment  with  which  apertures  are  cut  and  mea¬ 
sured  all  provide  clues  to  the  values  of  parameter  variances. 

Tlie  coefficient  values  ana  the  parameter  variance  values  can  be  inserted  in  the  equation 
for  n2^|j  to  estimate  reproducibility. 

While  an  analytical  expression  has  been  developed  for  the  blackbody  experiment, 
several  contributing  effects  have  not  been  treated  explicitly,  such  as  alignment  and  beam 
strength  across  the  collimated  beam.  Also,  second  order  and  higher  terms  have  been  dis¬ 
carded.  For  an  operating  aircraft  the  complete  specification  of  the  variance  associated  with 
reproducibility  would  be  extremely  difficult,  if  not  impossible.  However,  several  terms 
might  be  identified  which  would  contain  the  errors  contributing  most  significantly  to 
variance,  and  these  terms  could  be  quantified. 


CONCLUSIONS 


UNCERTAINTY  OF  DATA 

FROM  ONE  MEASUREMENT  SYSTEM 

There  are  many  physical  causes  for  uncertainty  among  measurement  data  collected 
by  the  same  measurement  system.  For  convenience  in  analyzing  and  characterizing  this 
uncertainty,  these  causes  may  be  divided  conceptually  into  three  categories: 

I  Reproducibility  of  the  experiment 

2.  Measurement  system  bias 

3.  Random  distribution  of  measurement  error. 

In  theory,  the  reproducibility  of  the  experiment  can  be  quantitatively  determined 
from  the  defining  equation  of  the  experiment  and  an  intimate  knowledge  of  the  measure¬ 
ment  processes  by  which  the  defining  parameters  for  the  experiment  are  evaluated.  In 
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practice,  the  defining  equation  or  sequence  of  equations  may  be  difficult  to  specify  com¬ 
pletely  for  complex  configurations.  Nevertheless,  even  a  partial  representation  of  a  complex 
experiment  may  be  analyzed  to  quantify  the  most  significant  contributors  to  reproducibility. 

Measurement  system  bias  is  the  most  difficult  of  the  three  categories  to  deal  with 
analytically.  While  no  deterministic  techniques  have  been  developed,  some  clues  about 
this  bias  may  be  derived  from  applying  a  statistical  test  of  significance  to  measurement 
data  and  subsequently  performing  a  qualitative  inspection  of  the  data. 

The  probabilistic  distribution  of  random  measurement  error  is  the  conceptual  area 
which  may  he  subjected  to  the  most  detailed  mathematical  evaluation.  Statistical  analysis 
of  the  measurement  data  may  be  performed  to  quantify  data  variability  for  a  range  of 
physical  conditions.  Numerical  analysis  may  then  be  applied  to  characterize  data  variability 
as  a  function  of  the  physical  property  evaluated  by  the  measurement  system. 


UNCERTAINTY  OF  DATA 
FROM  DIFFERENT  FACILITIES 

The  uncertainty  which  exists  among  data  collected  by  two  different  measurement 
systems  includes  all  the  effects  described  in  the  preceding  section  compounded  for  both 
facilities.  In  addition,  any  discrepancies  between  the  measurement  processes  used  for 
evaluating  the  defining  parameters  at  the  two  facilities,  such  as  bias  in  measuring  the  phys¬ 
ical  conditions,  will  contribute  an  additional  source  of  uncertainty. 


CHARACTERIZATION  OF  THE 
PROBABILISTIC  DISTRIBUTION 
OF  MEASUREMENT  ERROR 

The  major  goal  of  this  effort  is  to  characterize  the  probabilistic  distribution  of  measure¬ 
ment  error  for  the  Michelson  interferometer  measurement  system.  Based  on  the  limited 
matrix  of  physical  conditions  available  for  this  study,  a  mathematical  model 

=  p2(H  -  B)2  +  <;^2 

has  been  suggested  to  describe  the  variance  of  this  probabilistic  distribution.  Here 
P  is  a  coefficient  to  be  determined  by  analysis 
II  is  the  irradiance  due  to  the  source  at  the  collecting  optics 
B  is  the  mean  of  the  noise  distribution 

oN2  is  the  variance  of  the  distribution  of  noise  inherent  to  the  measurement  system. 
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Although  insu indent  data  are  available  to  yield  conclusive  results,  the  coefficient  p 
appears  to  be  a  function  of  distance.  The  constant  values  of  p.  It.  and  arc  characteristic 
of  the  particular  equipment  configuration  used  lor  measurement  data  collection,  and  must 
be  reevaluated  when  a  measurement  system  is  reconfigured. 


PERCENTAGE  VARIATION 
OF  MEASUREMENT  SAMPLES 

The  probabilistic  distribution  of  random  measurement  error  is  characterized  by  the 
standard  deviation  of  the  measurement  population.  For  this  particular  study,  the  measured 
parameter  is  irradiance  at  the  collecting  optics  of  the  measurement  system.  For  irradiance 
values  in  excess  of  1 500  x  10"'®  watts/cni“,  the  standard  deviation  of  any  sample  of  irradi¬ 
ance  measurements  for  the  interferometer  systems  studied  in  this  effort  may  be  estimated 
from  the  expected  value  of  irradiance,  using  the  percentage  variability  factors  given  in 
Table  13. 


Table  13.  Percentage  Variability  Factors. 


Facility 

Distance  from  source, 

m 

Percentage  variability 
(dimensionless) 

China  Lake 

30.48 

.24'^ 

60.96 

1.25% 

152.4 

1.65% 

White  Sands 

155.0 

.7% 

To  estimate  the  percentage  variability  for  expected  irradiance  values  closer  to  the  system 
noise  level,  the  graphs  given  as  Figures  18,  19.  20.  and  21  should  be  used. 


RECOMMENDATIONS 


UNDERSTANDING  THE  PROBLEM 

As  a  result  of  their  participation  in  the  test  program,  personnel  interviews,  and  research 
of  the  published  reports  in  the  IR  community  conducted  for  this  effort,  the  authors  have 
become  aware  that  the  problem  of  data  uncertainty  is  not  well  understood  in  IR  appli¬ 
cations.  This  same  problem  has  been  extensively  researched  and  discussed  for  other  disci¬ 
plines  in  particular  for  electrical  engineering  applications.  The  primary  recommendation 
of  this  report  is  that  the  measurers  and  users  of  aircraft  IR  signature  data  be  made  aware 
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of  the  nature  of  the  problem.  First,  an  understanding  of  the  causes  of  measurement  data 
uncertainty  would  alleviate  the  political  problems  resulting  from  observed  discrepancies 
between  results  from  two  different  measurement  facilities.  Second,  given  the  ability  to 
assess  the  problem,  every  measurement  facility  could  publish  its  data  with  qualifying  measure¬ 
ment  uncertainty  brackets. 

Assessment  Capability 
for  Measurement  Facilities 


The  recommendation  is  made  that  all  facilities  which  measure  aircraft  IR  signatures 
routinely  apply  data  analysis  techniques  such  as  those  described  in  this  report  to  their 
measurement  data.  These  data  analysis  algorithms  could  be  implemented  as  operational 
computer  programs  at  the  facilities  so  that  no  additional  manpower  would  be  required 
over  present  facility  staff  levels. 

Characterizing  the  Probabilistic 
i  Distribution  of  Measurement  Error 

When  the  blackbody  experiment  reported  herein  was  designed,  the  nature  of  the  1R 
measurement  uncertainty  problem  was  not  completely  understood.  The  experimental 
design  was  intended  to  gather  a  well-distributed  matrix  of  data  for  analysis  to  obtain  an 
understanding  of  the  problem.  In  fact,  the  distinction  between  the  reproducibility  of  the 
experiment  and  the  probabilistic  distribution  of  measurement  error  was  not  recognized. 
A  conceptual  division  of  uncertainty  for  one  measurement  system  into  three  catceories 
has  been  suggested  and  a  mathematical  model  proposed  for  describing  the  probabilistic 
distribution  of  measurement  error.  Well-qualified  data  should  be  collected  for  the  sole 
Purpose  of  verifying  or  improving  this  model.  This  effort  should  include  extensive  data 
in  the  low  signal  region  to  facilitate  the  analytical  determination  of  system  noise.  In  ad¬ 
dition,  a  sufficiently  detailed  range  of  parameter  variation  should  be  studied  to  determine 
the  functional  dependence  of  the  irradiance  coefficient  p*  The  matrix  of  data  gathered 
should  include  a  parametric  variation  over  field-of-view  values  to  determine  what  effect, 
in  any.  that  factor  has  on  the  values  of  the  constants  p.  R,  and  in  the  proposed  model. 
Above  all,  in  any  future  data  collection  efforts  the  editing  process  at  the  measurement 
facilities  should  be  supervised  by  the  data  analysts  who  will  be  doing  the  subsequent  model 
verifications. 

Studying  Other  Infrared 
Measurement  Systems 


When  a  verified  mathematical  model  for  the  characterization  of  the  probabilistic 
distribution  of  measurement  error  for  the  Michelson  interferometer  has  been  accepted, 
the  other  commonly  used  IR  measurement  systems  should  be  studied.  Radiometers  and 
spectrometers  arc  recommended  for  the  next  effort.  The  problems  of  mathematical  charac¬ 
terization  of  data  variability  for  imaging  devices  will  undoubtedly  be  much  more  severe. 


*Soc  Computing  Percentage  Variability  and  System  Noise. 
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and  a  background  of  experience  of  analysis  of  the  more  easily  described  systems  should 
be  acquired  before  attempting  to  study  image  data 

Summary 

The  reader  is  cautioned  that  the  data  variance  model  proposed  in  this  report  has  been 
developed  from  a  relatively  small  data  base,  and  that  additional  research  may  result  in 
modification  or  replacement  of  this  model.  This  report  is  offered  as  one  step  in  the  con¬ 
tinuing  effort  conducted  by  the  J IRS  Working  Group  to  improve  the  quality  and  usefulness 
of  aircraft  IR  signature  measurement  data. 
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APPENDIX 

STATISTICAL  DEFINITIONS 

l  or  a  sample  of  measurement  data  containing  N  items.  |xj,  x^.  Xj  .....  x^  |.  the  Rth 
moment  of  the  data  about  a  value  A  is  defined  as 

N 

E  (xj  -  a)r 

i=l 

N 

The  first  moment  of  the  data  about  the  origin  (A  =  0)  is  defined  as  the  arithmetic  mean  x. 
This  value  x  has  the  property  that  the  first  moment  of  the  data  about  the  arithmetic  mean 
is  0.  The  second  moment  of  the  data  about  the  arithmetic  mean  determines  the  sample 
variance  S2.  The  square  root  of  this  value  S  is  the  sample  standard  deviation,  a  measure 
of  the  dispersion  of  the  data.  Formulas  for  the  first  four  moments  for  sample  data  are 
given  in  the  following  chart. 


Moments  about 

Moments  about 

Moments  about 

an  arbitrary 

the  origin 

the  arithmetic  mean 

point  A 

(A  =  0) 

(A  =  x) 

First  moment 

N 

N 

N 

E  <xj  -  A) 

i=l 

E  xi 

i=l 

E  <xj  -  X) 
i=1  -  0 

N 

N 

Second  moment 

N 

N 

N 

E  <xj  ‘  A)2 

E  xi2 

E  <xi  -  x>2 

i=l 

i=l 

i=l  N  -  1 

N 

N 

N  N 

Third  moment 

N 

N 

N 

E  <xj  -  A)3 

i=l 

EV 

i=l 

E  <xi  -  x)3 

i=1  M  ' 

N 

N 

N  "  M3 

Fourth  moment 

N 

N 

N 

E  <Xj  -  A)4 

E  xi4 

E  <xj  -  x>4 

i=l 

i=l 

i=I  -  M  ■ 

N 

N 

N  "  M4 
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The  moment  coefficient  of  skewness  is  defined  by 


For  a  normal  curve,  =  0. 

The  moment  coefficient  of  kurtosis  is  defined  by 


Fora  normal  curve,  a^  =  3. 
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